Monday, May 26, 2014

Nerd Food: Using Mono In Anger - Part I

Nerd Food: Using Mono In Anger - Part I

In which we convince ourselves that it is time to use Mono in anger.

I've always been a Mono fan from the early days. Sadly, my career as a Mono developer never amounted to much - not even a committed patch - but I remained a loyal lurker and supporter. It's not so much because I thought it was going to change the face of Free Software, but because I wondered if it would allow Linux to gain a foothold in Windows-only companies. Having been working for this lot for such a long time, I desperately yearned for a bit of serious Linux interaction in my professional life, so I kept on playing around with Mono technology to evaluate progress. For instance, a few years back I wrote Adventures in F# Land Part 1, Part 2 and Part 3, based on my experiences in setting up F# in Mono1. I continually did this kind of mini-experiments, mucking around with tiny programs to query the state of the world. But what I really wanted to do was to test the whole infrastructure in anger2.

As you can imagine, I was extremely excited when I read Miguel's tweet:

Microsoft has open sourced the new generation of http://ASP.NET http://github.com/aspnet

Another blog post had more details: ASP.NET vNext. Microsoft was to start testing code against Mono, at least for ASP.Net. To me this was nothing short of revolutionary. Not to the average Linux developer, of course - barely a yawn from that camp. But to the average .Net developer this is - or should be - earth shattering news. When you couple this with the .Net Foundation announcements, it signals that Microsoft now gives the official blessing for developers to start thinking of .Net as cross-platform.

It may surprise you, but the average .Net developer does not spend days and nights awake, worrying about how to carefully design their architecture to maximise cross-platform code. They do not have Mono on their CIs. They tend not to care - or even be aware of - all the latest and greatest features that the Linux kernel sports, or how cool Docker, Chef and Puppet are. Those that are aware of this are in a minority, and most likely have been exposed to the other side via the Javascript infiltration that is now taking place. A fairly recent comment in a Castle issue should give you a flavour of this thinking:

I don't think anyone is actively pursuing Mono support at the moment. Reality with Mono historically has been that no one cared enough to properly bring it to parity with .NET and keep it that way.

Having said that, we're more than happy to improve our Mono support (…)

For those not in the know, Castle is an Open Source project that provides an IoC (Inversion of Control) container. It is really popular in .Net circles. As we shall see later, Castle actually works really well with recent versions of Mono - but the fact that the Castle team didn't have a CI with Mono is indicative of the state of the world in Windows .Net.

This is not to say that Linux is not on the Windows developer's radars - quite the opposite, and the progression has been nothing short of amazing. During my working life I've spent around 70% of my time working for Windows shops; to be honest, I still find it surprising when look at the past and compare it to the present. Things went from "Linux, what's that?" to "Linux is a nerd's toy, it will never make it" to "That's for Java guys" and finally, to "We run things on Linux where it is good at, but not everything". Things being Java and Coherence, Grid Engines, Javascript stacks and so on - no mention of .Net.

In fact, the .Net revolution hampered the Linux strategy a lot. Java was a good Trojan horse to force developers to think cross-platform, and many a Windows shop turned to Linux because of Java. However, once you had a competitive (read "enterprise-ready") .Net framework, Java receded in Windows people's minds and it was back to business as usual. Even a strong and capable Mono was not enough; a lot of the .Net code was so incredibly intermingled with Windows-only code that portability was impossible without great effort. Many a hero tried and failed to compile massive code bases on Linux.

Things started to change over the last five years. Three separate trends unified. First was the new focus on SOA. Windows developers finally found out what the UNIX geeks knew all along: that services are the way forward. And because their view of SOA was closely linked to REST and Web development, suddenly the code bases became a lot less Windows dependent and a lot more self-contained; no more WPF, no more third-party closed-source controls. The second important development was that Windows developers found Free and Open Source Software. A lot of the libraries they rely on are open source, and thus much more Mono friendly. The final trend was the rise and rise of Javascript and Linux-only tools. Every other Windows developer is now familiar with Node.js, Mongo, Redis et al, and have at least rudimentary knowledge of Linux because of it.

However, the Linux developer on a Windows-land is still stuck, and all because of a tiny little detail: IIS and ASP.Net are still a pain in the backside. Mono has provided ASP.Net support for a while now, but to be honest, the applications I've seen in the wild always seem to use some features that are either not yet supported or not supported very well. There is just so much coupling that happens when you pull in IIS that is hard to describe, and a lot of it happens without people realising - it's that lack of cross-platform mentality all over again. So for me the one last little piece of the puzzle was IIS and ASP.Net. This was anchoring the entire edifice to Windows.

Now you can see why having a fully open source ASP.Net stack is important; especially one that does not depend on IIS directly, but uses OWIN for separation of concerns and is tested on Mono by Microsoft. Code bases designed on this stack may be able to compile on Mono without any changes - or, more likely, with a few small changes - opening the floodgates for Linux and OSX .Net development. Of course, don't let me get too carried away here. There is still much, much to be done for a totally cross-platform environment: Powershell, KPM, etc. But this is a very large step in the right direction, and it covers a large amount of use cases.

A final point is worth making about the corporate ecosystem that evolved around Mono. This is not really a marketing ploy, although I do wish them all the best. Companies such as Xamarin, Unity and projects such as MonoGame are instrumental in the progress of Mono. By creating funded start-ups around Mono, with products that are used by a large number of developers, they engineered a dramatic increase of quality in the entire ecosystem. And in turn this raised its visibility, bringing more developers on board, in the usual virtuous circle of open source.

So for all these reasons and more, it seemed like Mono deserved another good look. As it happened, I was trying to develop a fairly large test tool for work in my copious free time. Since I needed to code from home, I thought this would be the right project to give it a far old whack. And you, dear reader, will get to know all about it in this series of posts.

Footnotes:

1 Mind you, all of it is to be ignored as its now trivial to setup F#.

2 For the non-British speakers out there, using something in anger just means "to give it a good go" or "to really push it" rather than writing a hello world.

Date: 2014-05-26 17:24:40 BST

Org version 7.8.02 with Emacs version 23

Validate XHTML 1.0

Sunday, May 26, 2013

The Beautiful Game

The Beautiful Game

The Glory Days

When I was a kid I was crazy about football. And I don't mean just the average "crazy", I mean really crazy, as in totally nuts. These days they'd probably give me some sedatives and call it something like OCD; in those days my dad used to say we had the febre do futebol or football fever. Me and my older cousin Bruno, man, we were dedicated fans. Bruno's dad used to bring the A Bola football newspaper every Thursday - I don't think it was a daily paper back then - and we used to read it from cover to cover until the next edition came in. We watched any and all football matches on TV we could get our hands on.

At the age of eight or nine I could tell you all the players in all the squads in the Portuguese premier league (the first 25-odd) and so could my cousin. We knew most of their ages and a lot of their transfer history. We could name most players of the second division. We could also roll-call first-elevens and in some cases entire squads from all the major European teams in leagues such as English, French, Spanish, German, Italian, Dutch and many others I now forget - for instance I remember knowing all of the players of Rapid Vienna, as well as the useful fact that Rapid is one of the few teams in Europe to have won their own league (Austrian) as well as the German league.

And we lived the games. Neuchatel Xamax versus Zurich was a showstopper. We endlessly replayed the Red Star Belgrade versus Partisan Belgrade derbies as if we were in the stadium, and cried over Celtic versus Rangers as if we were raised in Glasgow. Few people knew the entire Anderlecht squad, but for us Anderlecht, Brugge, Malines - these were household names. We'd laugh at people that confused Cercle Brugge with Club Brugge. But wait, it doesn't end there. We could name the sixties squads for Real Madrid, Benfica, Manchester United and a few other teams. Puskas was my hero, and so were the other Hungarians. We saved our meagre allowances to spend on the yearly Portuguese league sticker collection, and never ever missed the European Cup and World Cup sticker collections.

At some point in our teenager years our friend Marco joined our crazy football-mad-nerd-gang, and if anything things got even more intense. We discussed tactics endlessly, coaches, schools of thought. And one day, Marco had the brilliant idea to go and actually watch Vitoria Futebol Clube (VFC) live. As in, at the stadium. In loco. In those days money was tight, so it never even had occurred me that you could go and watch live games. So we saved for ages, convinced our parents to allow us to trek to Setubal and off we went to watch a game. I don't even recall what game it was, I just remembered being in awe of a full Bonfim stadium with lots and lots of VFC fans. In Portugal, only Porto, Benfica and Sporting had fans; the rest had empty stadiums. VFC was one of the exceptions, in those long forgotten days.

I have been a VFC fan since that day.

The Dark Days

On hindsight, when I look back I now think we were a bit like football savants; we were truly, deeply, profoundly in love with the game but - perhaps due to our young age - we didn't have the intellectual depth to make proper sense of it all. We weren't skilled enough as players - I certainly was one of the worst footballers to have graced a football pitch - and we didn't have the tenacity of a young coach - to go out there and learn proper tactics, to go on courses make a career of it, even if part time.

Many, many years later, when I met Jonathan Wilson - he of Inverting the Pyramid fame - and many other reporters I finally understood what it would have meant to have both the insane passion and the drive to obtain the intellectual depth required to be a part of the game rather than yet another spectator. As it was, I soon discovered computers and shared my love for football with programming for a decade or so - less and less football, more and more computers. Until University, at which point I just stopped paying attention to football altogether.

When time came to write a bachelor thesis I suddenly thought, here's the chance to right all the wrongs and finally learn about the football business. I spent several months immersing myself in Futebol Clube Barreirense (FCB), my local club. VFC was just too far away to commute, so FCB was the next best thing I could find. The objective of my thesis was very simple: to understand why once great clubs decline; and to try to apply all the newfangled business and marketing techniques to figure out if there was anything to be done for FCB.

After a good few months of hard work, what I found was profoundly disappointing: FCB had been a major institution in years gone by, but the decline was deep. Barreiro at one point had two major clubs, FCB and CUF - both of which were fighting for European places in the seventies. CUF was pretty much dead and buried, great at the schools level but non-existent at the over-21's level. FCB was fast joining them, just about surviving. Eighty percent of their core supporters were aged over seventy and the fan base was literally dying out and not being replaced. The club was shrinking. Being a small club had its advantages; I had direct access to the president and he helped me understand the history of the club, and to imagine a future. I did endless treks to schools in the district to try to figure out why it was that none of us bothered to turn up and watch games when the cost was pretty minimal. My "environmental analysis" revealed that Barreiro and its surrounding areas was not a small place; getting ten thousand people to watch a match should have been doable - and yet they were getting a hundred spectators on good match days.

Fundamentally, what I learned was that there was nothing wrong with football per se; with a lot of effort one could try and turn around the FCB brand. But the biggest problem was organisational - what they called "execution" in the University management circles. You just did not have the right mentally to execute; it was pervasive at all levels of the Portuguese football machine, but more so in the little clubs. All my hundreds of pages were dutifully read by club management - or so I hope - and quickly filled away into the "waster of time" section in the cupboards.

The Present

I'm a rather sceptical football fan these days. I pretty much lost touch with football in general and specially Portuguese football - other than occasionally following the news on VFC. The other day I went to watch Peterborough v Brighton live and saw some ten thousand fans supporting a team fighting for relegation on the second tier of English football. I could not help but be very saddened that my once great VFC now had a lot less fans than this club on a good day.

If anything, the "Super Liga", the Portuguese answer to the premiership, has failed small clubs even more than the previous setups. Whilst very organised teams like Braga are on the way up, everyone else seems to be shells, there to provide the occasional upset but little else.

Benfica may be fighting for the UEFA cup, and Porto may be winning the League - and to my great surprise, Guimaraes may even win the Portuguese Cup. But the Portuguese league is just a three-way version of the Scottish league, a few giants and lots of minnows just barely surviving.

Date: 2013-05-26 21:59:08 BST

Org version 7.8.02 with Emacs version 23

Validate XHTML 1.0

Sunday, May 19, 2013

NP: Yá Yá Massemba


Que noite mais funda calunga
No porão de um navio negreiro
Que viagem mais longa candonga
Ouvindo o batuque das ondas
Compasso de um coração de pássaro
No fundo do cativeiro
É o semba do mundo calunga
Batendo samba em meu peito
Kawo Kabiecile Kawo
Okê arô oke
Quem me pariu foi o ventre de um navio
Quem me ouviu foi o vento no vazio
Do ventre escuro de um porão
Vou baixar o seu terreiro
Epa raio, machado, trovão
Epa justiça de guerreiro
Ê semba ê
Samba á
o Batuque das ondas
Nas noites mais longas
Me ensinou a cantar
Ê semba ê
Samba á
Dor é o lugar mais fundo
É o umbigo do mundo
É o fundo do mar
Ê semba ê
Samba á
No balanço das ondas
Okê aro
Me ensinou a bater seu tambor
Ê semba ê
Samba á
No escuro porão eu vi o clarão
Do giro do mundo

Que noite mais funda calunga
No porão de um navio negreiro
Que viagem mais longa candonga
Ouvindo o batuque das ondas
Compasso de um coração de pássaro
No fundo do cativeiro
É o semba do mundo calunga
Batendo samba em meu peito
Kawo Kabiecile Kawo
Okê arô oke
Quem me pariu foi o ventre de um navio
Quem me ouviu foi o vento no vazio
Do ventre escuro de um porão
Vou baixar o seu terreiro
Epa raio, machado, trovão
Epa justiça de guerreiro
Ê semba ê ê samba á
é o céu que cobriu nas noites de frio
minha solidão
Ê semba ê ê samba á
é oceano sem, fim sem amor, sem irmão
ê kaô quero ser seu tambor

Ê semba ê ê samba á
eu faço a lua brilhar o esplendor e clarão
luar de luanda em meu coração

umbigo da cor
abrigo da dor
a primeira umbigada massemba yáyá
massemba é o samba que dá

Vou aprender a ler
Pra ensinar os meu camaradas!

Vou aprender a ler
Pra ensinar os meu camaradas!

Saturday, April 27, 2013

Nerd Food: Interesting...

Nerd Food: Interesting…

C++

Emacs

  • Google Reader alternative for Emacs Ninjas: So the Mayans were almost right, the world is going to end in 2013, the year when Google reader will be decommissioned. Everyone is looking for ways of averting the catastrophe - I've even voted in one or two petitions. This guy proposes using Emacs. Hey, might give it a go, doing everything else on Emacs so why not?
  • Rootless Root: The Unix Koans of Master Foo: Missed this one from ESR somehow. Very jargon'esque, old school koans from the lisp days.

Other Programming Topics

Finance

  • Behavioral Finance Explains Bubbles: Pretty much everyone is convinced that regular economics has failed; its good to see that more attention is being paid to interesting offshoots such as behavioural economics. These guys start with how people actually behave rather than how they ought to behave, so their models have a fighting chance.
  • Crushing national debts, economic revolutions, and extraordinary popular delusions: Everyone has been on top of Rogoff and their excel error (which reminds me, you could do worse than listening to NPR's planet money take on it: Episode 452: How Much Should We Trust Economics?). Andrew Odlyzko looks at the problem from another angle, namely how Britain looked like around the industrial revolution. I always struggle with these historical extrapolations but he does make an interesting case.
  • The Bitcoin Bubble and the Future of Currency: Everyone has written an article or two on Bitcoin of late. This is yet another one, with an intro and an explanation on why it will never catch on. Some interesting points, despite shortsightedness.

Other

Date: 2013-04-27 18:49:27 BST

Org version 7.8.02 with Emacs version 23

Validate XHTML 1.0

Wednesday, April 24, 2013

NP: Santa Maria da Feira



Pensando cada dia, cada hora
Pensando en ti
Caminando, mi cesta llena de moras
Son para ti
Temprano por la tarde y por la noche
Sueño de ti

Lalalala

Comiendo pera
En santa maria de la feira
Que placer
La gente buena solo goza nunca hay pena
Pa' que sufrir
Jugando en el mar, en la arena
Viviendo asi

Lalalala

Ventana blanca
Ay, que venga la mañana
Ay que venga otra vez
Esperando
Asi es como yo paso mi tiempo
Esperando a Inaniel
Y rezando por su calor, por su aliento
Sobre mi piel

Te digo todo aqui va bien
Conmigo de no dormir
Amigo, te lo suplico, te lo pido
Que me ayudes a mi, a mi

Buscando
Con mi ancla en la marea
Nadando en ti
Yo voy andando
Oyeme, te estoy llamando
Te amo a ti

Por el valle me encontré un rio escondido
Me recuerdo, hacía calor pero tenia frio
Me iba a morir

Bianca
Ay paloma angelina
Por fin te vi



Sunday, April 21, 2013

Nerd Food: C++ times, they are a changin' - Part I

Nerd Food: C++ times, they are a changin' - Part I

I think most C++ developers would agree that there has never been a better time to be a C++ developer than the present: after all, the C++ 11 standard was released adding a slew of modern features to the language and we are now seeing clang and gcc nearing 100% feature completion. In addition, there is an amount of activity around the language and libraries that I don't ever recall seeing in my professional career; the C++ 14 papers give a pretty good idea of the diversity and reach of this work.

In effect, C++ is negotiating the transition from a closed, vendor driven platform to an open, community driven project. These two articles will attempt to narrate the process, from the perspective of a long time C++ user.

From the beginning, Stroustrup had always believed that without libraries C++ would be a pretty useless language:

Without a good library, most interesting tasks are hard to do in C++; but given a good library, almost any task can be made easy.

Plain C++ was a pretty useless language: many developers reinvented the wheel, and libraries of poor quality were endlessly rewritten. Its hard to remember now, but in the nineties it was a rite of passage to write your own containers such as doubly-linked lists. Slashdot user VortexCortex gives a pretty insightful picture of those long forgotten days:

Well, I wrote my own Hash Map implementation. Before that I had my own Linked Lists too. Before C++ came along I even maintained my own OOP implementation in C with a spiffy pre-processor to convert syntactic sugar into the equivalent ugly object oriented C code riddled with namespace prefixes, indecipherable pointer math for dynamic (virtual) functions (actions), and statically complied variable (property) look-up tables for run-time type inspection (reflection).

It led to incompatibilities between codebases – My Entities+C wouldn't be compatible with your C+OOP. Hell, we didn't even use the same terminology, or necessarily even support the same feature sets. The point is, I wasn't the only one who was doing that (see also: Objective C). There were lots of us, it was a huge waste of duplicated effort. So many, in fact, that C++ was born, and the STL too […].

It is in this context that the C++ 98 standard emerged, and it was seen as a major victory for all involved - a milestone in the development of the language. However, pragmatism won the day and parsimony is always a requirement for shipping; the standard library was pretty much just the STL, the I/O Library and naively-defined string support. Once one exhausted the core functionality of the standard library, well, you were on your own. This is were library vendors came in.

C++ had always been a highly profitable tooling market for vendors, and the vacuum of specification only served to amplify the profits. It was now possible to make a true claim for vendor independence - "hey we are 100% standards compliant!" - while at the same time providing large amounts of vendor specific code. Once a vendor was chosen, vendor lock-in was almost by design: the libraries provided so much functionality and in such vendor-specific ways that changing libraries was synonymous to rewriting entire code bases.

The other interesting aspect of these days was that C++ community just didn't know how to write good C++. Most of the vendor libraries coped out and simply defined object oriented APIs because that was easier to create and easier to consume. Stepanov's dreams - that the STL was simply one example of many of a different way of thinking, and a raft of libraries along the same lines were soon to follow - were put on hold.

But people were already learning from the success of community driven languages such as Perl and Python. A man that was ahead of the curve in this regard was Beman Dawes. Not only did he deeply understand the standardisation process - and its flaws - but he also saw that libraries were the key to success. Thus Boost began as an effort to provide the missing libraries, in a nimble, community driven way that at the same time had high quality standards; these were provided by peer reviews, requests for documentation and so on. Boost soon became the home for the "fundamental" C++ thinking, in many ways continuing the discovery process of the Stepanov ideas in libraries such as Boost.Graph.

In the mean time, the world didn't stand still. The rich pickings C++ provided to vendors proved to be their downfall, as the largess of the tools market gave way to lean times. The crowds moved over to the new shinny toys such as Java and C#, and the new breed of dynamic languages gave C++ the final kick into the legacy bucket. It was just not cool anymore to be seen doing C++. These new languages learned their lessons and came out of the box with large class libraries that provided support for all sorts of things, making the standard library look small and antiquated in comparison. Even with Boost, one would still be nowhere near the functionality of out-of-the-box Java or C#, let alone Python or Ruby.

When a COBOL-style death appeared certain, a phoenix-like effect kicked in. Just as it was with UNIX - where the death and shrinkage of UNIX vendors became the catalyst to create a new breed of UNIX based on collaboration rather than competition - so the same happened to C++. When the big money moved away it was suddenly noticed that no single company could afford to maintain a large library by themselves on the skimpy margins they were now making. Thus many companies opted for open sourcing their code - some just before they went out of business, others as a strategic way of staying in business. Many vendors died, but the few that remained found healthy niches from which to run their businesses.

The second factor that kept C++ alive after all the cool kids moved on was that the old school engineers were still around, and when efficiency became a key concern - as it always does in the end - they were there to explain how efficient server side code could be written in C++. So whilst uncool, C++ was alive and well, just surviving underground in the guts of data centres and in large scientific projects. These users were making hefty use of open source, with Linux GCC and a plethora of open source libraries as their weapons of choice. Together with the appearance of Clang, this mean that both the tooling and the libraries in C++ moved forward in leaps and bounds, but in a process that was almost entirely divorced from the standard. From not having enough libraries we went to having too many of them, with lots of overlapping functionality.

Whilst this was happening, C++ didn't stand still. Some of the more obvious Boost code got shipped into TRs and made their way to end users. And C++ 0x was cooking. This was a major change to the standard, composed both of core language changes but much more importantly, of a great deal of additions to the standard library. A number of Boost libraries were polished up and moved into the std namespace.

But this brought with itself its own problems. Some were related to the new language features - its still not certain how some of these should be used - and others are related to the new libraries. For those of us heavily reliant on Boost, a difficult problem emerged: the standard libraries didn't cover all aspects of the Boost libraries they standardised; for instance, Boost Serialisation won't work with std::shared_ptr, so now code bases become a mix of Boost and standard library code. Clearly the process of offloading Boost libraries to the standard has only begun. And this is what we will be covering in part II.

Date: 2013-04-21 19:02:43 BST

Org version 7.8.02 with Emacs version 23

Validate XHTML 1.0

Thursday, April 18, 2013

Nerd Food: Restoring a file from Git history

Nerd Food: Restoring a file from Git history

A common problem I've had with git in the past is how to restore a file from history. As it turns out, like with many things in git, it is quite easy once you know what you're looking for.

First we need to locate the file. For this we look at all paths affected by the commits we have done over time, so git log is a perfect suitor:

$ git log --all --name-only --pretty=format: | grep -i ecore
diagrams/other/Ecore.dia
diagrams/Ecore.dia
diagrams/other/Ecore.dia
diagrams/Ecore.dia
doc/diagrams/Ecore.dia
doc/diagrams/Ecore.dia

As I'm not really bothered as to which commit did what to this file, I just picked one of these paths. You may need to look into history and try to figure out which version is the one you're after. Now we need to figure out the SHA key:

$ git log -- diagrams/Ecore.dia | grep commit
commit f2d763df8b3775a9370ed43e00e3c28a07190932
commit a183879d848f32cd0f8b14c3b4b707f2b4946ca5

I'm sure you could do both of these steps in one go, but I'm too lazy to find out how. Finally, just checkout the blob from the object database:

$ git checkout a183879d848f32cd0f8b14c3b4b707f2b4946ca5 -- diagrams/Ecore.dia
$ head diagrams/Ecore.dia
<?xml version="1.0" encoding="UTF-8"?>
<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/">
  <dia:diagramdata>
    <dia:attribute name="background">
      <dia:color val="#ffffff"/>
    </dia:attribute>
    <dia:attribute name="pagebreak">
      <dia:color val="#000099"/>
    </dia:attribute>
    <dia:attribute name="paper">

It's that easy!

Date: 2013-04-18 23:51:11 BST

Org version 7.8.02 with Emacs version 23

Validate XHTML 1.0