The rust programming language announced the release of Cargo, their new package manager, which will, "support the common lifecycle for packages", because we don't have enough of that already.
Before I start, let me just say - I'm not picking on rust or cargo. This isn't about them. This is about the ecosystem.
We have too many package managers. Far, far, far too many package managers.
Each Unix-like OS has their own package management solution. And each programming language seems to have ITS own, too. Whether it's CPAN, or pip, or pear, or easy_install, or gems, npm, or now cargo, there's a way to install addons for software specific to a language, in that language itself, but more importantly, outside of the OS package management.
And then, there are other layers to contend with. DSLs have their own internal way of installing modules. Puppet has the forge, chef has knife and the community cookbooks, and CFengine has the..well, whatever it has, I'm sure. Then there is meta-software to manage those, too. Things like librarian-puppet, which is, itself, a customization of a piece of software called librarian, which is a framework to manage bundlers, which is something that manages ruby dependencies (pulling down the software you need to run the things that you have). There is also librarian-chef. This is separate from Berkshelf, of course, which does pretty much the same thing.
How in the ever-loving sight of his noodley appendages are any of us supposed to manage this heaping layer of crap?
Yes, I know, some of the aforementioned software solutions are meant for managing a single user's module directory maybe, or a specific dev environment, or for testing. But which, and why, and is it explained anywhere that the software should or shouldn't be used in a certain environment or way, or are you just given the sticks, the twine, and the plastic and told to go fly a kite?
The irony of all of this is that we're using this jumbled up pile of random software which largely overlaps in purpose and function, to try to build more solid, easily reproducible infrastructures. But rebuilding the environment that created it? Good luck!
Maybe it's not all that bad. Maybe. I mean, after all, I have the artifacts that were created, right? I have the puppet code that I used to produce my infrastructure. Well, sort of. I mean, to be really honest, I don't write all that much puppet code. I use off the shelf puppet code, and I end up writing YAML for hiera to interpret and feed into parameterized classes. So my artifacts aren't puppet code so much as really ugly stuff that isn't markup, according to the definition of the language construct.
So how do I rebuild the compost heap infrastructure that I used to build my environment? Suppose I used entirely off-the-shelf puppet code. Nothing custom, just modules I found. And I erased my repo which contains my puppet modules. How would I rebuild it and get the same thing that I had before?
Do I stand a chance in hades of actually being able to reproduce what I had before? Suppose I even had a list of all of the puppet modules. Could I source the same modules? The same versions of the same modules? No. And I'm willing to bet, neither could you.
If you're like me, you've probably cobbled together some kind of unholy array of modules from the forge, modules from github, and modules of ill repute that you found in a back alley somewhere, and that's all stuff that you probably think of as "off the shelf". It's completely irreplaceable, if only because there's no good way to figure out where what came from, because I (and most likely you) used any one of a dozen methods of getting the module into the system.
I don't think there's any hope for fixing the 'bespoke build infrastructure', at least in this generation. We have to deal with the fact that the systems we use to build our highly available, fully reproducible, solid infrastructures are, themselves, anything but.
However, if I could plead to the tool makers, to the developers who are, have, and will build the things we use to build things, let me ask you this. Please, swallow your hubris. Please, accept that a tool someone else wrote that you see as imperfect may actually solve the problem, and know that the world may not need another solution that does the same thing. Instead of building something new that overlaps, work with the other person to extend what they've done, don't build anew.
Unless we learn this lesson, we're going to continue to have infrastructures that look like the Walled City of Kowloon, rather than something better. Please.