This guy seems to be of the opinion that the software should be completely isolated from the deployment operating system.
I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the line? He seems to draw it further down the stack than most people (no system python, for instance) but he doesn't give his opinion on, for instance, using the system postgresql.
Anyway, I personally would draw the line further up the stack than him, but take things on a case-by-case basis, and I don't really consider it an "anti-pattern."
With regards to fabric vs. puppet, I understand the advantages of puppet when you have a complicated, hetrogenous deployment environment. But the majority of projects I've worked on have the operations model of a set of identically-configured application servers back-ended against a database server. For this configuration, what does puppet give you? If the author's argument is that the site may eventually outgrow that model, well, I can see puppet becoming necessary, but why not cross that bridge when you get to it?
I think there's an exageration as well, today the trend seems to be "use nothing by the distro"
Ok, sure, MongoDB still changes a lot between versions, in this case you should use the latest version.
But stop there. Especially if you're paying for support (like RHEL)
There should be a good reason for you to compile Apache / MySQL / PostgreSQL / Python. Otherwise, use the distro version. One (common) exception would be "we need Python 2.7 but this ships only 2.6"
Most of the "just download and compile" have no idea of the work that goes behind Linux distributions to ship these packages.
Yes, I'm sure you're going to read all security advisories and recompile all your stack every X days instead of running apt/yum upgrade
Yes you should, but if you have them: do it. That’s what the article says. ;) Please don’t push it in a wrong direction.
What I actually wrote is: because we’re a LAMP web hoster, we compile MySQL+PHP+Apache ourself. And because we’re a Python shop, we don’t let Ubuntu/Red Hat dictate which Python version we use.
If you use puppet/chef for the whole stack, you gain the ability of just starting a new machine and in a couple of minutes having it configured in the exact same way as all the others.
With fabric and similar systems it's a bit harder. Basically you'd have to write your scripts exactly the same way you'd write a puppet/chef recipe: "make sure this is configured that way, make sure that is installed", etc. (or do migration steps) It's very different from fabric's "do this, do that" approach. Unless you run fabric on every single host after you make every change, some of your infrastructure will be lagging behind.
For example, what do you do when you create a new server, or do an upgrade that involves different dependencies? Run a fabric script that migrates from state X to Y? What happens to new machines then? How do you make sure they're in the same state?
I found chef a very good solution even if I have a single server to manage. No need to think about how it was configured before. Migrating to another provider? Just migrate the data, point the server at chef, done.
Point of fact, he does give his opinion on the postgresql issue: "We also have to compile our own Apaches and MySQLs because we need that fine-grained control."
But that's besides the point. I don't think he's arguing that software should be completely isolated form the deployment operating system. That would be absurd, since, as you pointed out, software has to interact with the OS at some level, i.e., to manage system resources. Just because the OS ships with a bunch of packages with specific versions doesn't mean you have to use them. And what I think the OP is saying is that to make your applications portable and easily deployable, you shouldn't.
Good catch. Sorry I missed that. I suppose I've forgotten that a lot of people still use MySQL, even for python. :)
The question "why even have an OS?" was a rhetorical one, meant to illustrate the fact that the argument here is over where one draws the line, between the dependencies you maintain yourself and those you outsource to your OS vendor. Unless you're deploying on Linux From Scratch then you are outsourcing to the vendor at some level, so the only question is, where is that level?
That this guy maintains even his own web and database servers means wants/needs to control things at a much deeper level than is typical. And I think that's beautiful if it works for him. My point of disagreement is his use of the word "antipattern" to suggest that any other choice is wrong.
First of all I’m afraid the point about Apache & MySQL is misleading and I’ll have to clarify it: We don’t use neither for internal projects. It’s all PostgreSQL. :)
They’re both for our customers as we sell traditional web hosting with a LAMP stack (and yes, we have to compile PHP ourselves too, so we can offer different flavors).
I never said you _have_ to compile everything yourself. In case of Python I said you shouldn’t inflict the pain of programming Python 2.4 just because you’re on RHEL/CentOS 5 or earlier.
System packages have several problems I outlined too. Most important: you add a middleman to the release pipeline, no virtualenv and possible dependency conflicts: I have apps that depend both SQLAlchemy 0.6 and 0.7 – they couldn’t run on the same server.
But OTOH we use stock packages as far as possible, because it’s less work. Eg. there’s no reason to compile an own Python on Oneiric and later. Same goes for PostgreSQL which is uptodate.
It’s all about freeing yourself from constrains in points that matter, not adopting a new religion.
I know that's a common view and wrapping as much of the site as possible up in a virtualenv certainly has a lot of advantages. But ultimately, your software is going to have to interact with the OS, at some level, otherwise, why do you even have an OS? So the question is: where do you draw the line? He seems to draw it further down the stack than most people (no system python, for instance) but he doesn't give his opinion on, for instance, using the system postgresql.
Anyway, I personally would draw the line further up the stack than him, but take things on a case-by-case basis, and I don't really consider it an "anti-pattern."
With regards to fabric vs. puppet, I understand the advantages of puppet when you have a complicated, hetrogenous deployment environment. But the majority of projects I've worked on have the operations model of a set of identically-configured application servers back-ended against a database server. For this configuration, what does puppet give you? If the author's argument is that the site may eventually outgrow that model, well, I can see puppet becoming necessary, but why not cross that bridge when you get to it?