Music + Technology + Random Nonsense from the Music Industry by Ethan Kaplan, VP Product, Live Nation

Technology: Stacks, Platforms, Chaos and Anarchy

This last week my name got dragged into some blogs because of the folding of CIsco’s EOS unit. EOS was a hosted content management and community enablement platform that was adopted by Warner Music Group a few years ago.

At Warner Bros. Records we adopted the Drupal platform over four years ago as our content management and community system. The choice of Drupal came after I spent a weekend duplicating functionality that we were paying a fan club company for on both Joomla and Drupal. Out of this I decided Drupal was our best bet.

Over the last four years the team at WBR has made over 220 artist websites. At the peak of our Drupal operations we were launching nearly four a week. We were heavy open source advocates and very vocal about why we made certain technology choices. My team and I spoke on blogs and at conferences regarding this.

Drupal ended up forming the basis of our technology stack at WBR. This basis extended not just in how we made artist experiences online, but also to the methods by which we applied technology to the business at all levels.

Given the amount of press that went to the death of a music/artist platform last week, and my name getting dragged into it, I wanted to get some stuff about technology off my chest. This is in no way a testament to the success or failure of any platform or approach, just what guided me in my decisions and will so in the future.

Stacks and Platforms

Stack is a term that within computer science defines a linear dependency through layers of operational specialty. The most common use of it is the TCP/IP stack, which depending on the layer routes packets between routers, machines or applications. The important thing about the Stack concept is that layers at the top are dependent on the bottom.

Platforms are similar but more focused on creating an ontological framework for technology use and application. They provide normative processes, constraints and methods to conform to in order to allow the chaotic use of technology in an orderly way within a specific ontology. This is most readily seen on Facebook or in how you program for iOS, Mac OS or Windows, and especially video game platforms.

My approach to technology choices at WBR was oriented toward the application of a holistic stack in order to create a comprehensive platform for innovation..

There ends my diplomacy, here is my statement.

The stack as I defined it at WBR was as such:

Presentation/Interaction (Web, iPhone, etc) —> Drupal, WordPress, some Python (Tornado), iOS
Application Abstraction (authentication, analytics, etc) –> Digital Detail (custom data platform), Python, MongoDB, Janrain
API (third party integration) –> Digital Detail, Python (Tornado)
Data (consumer and operational) –> MongoDB, MySQL, Hive
Media (streaming, download, etc) –> Akamai, Brightcove, etc

Given this stack, which I fought diligently to maintain against the tide of alternatives, here are my loose pronouncements in terms of the application as such in a record company.

1. If your company uses the word “tech” in a pejorative sense, it’s best to pack it up and call it a day

With the use of Drupal initially we had assume tasks related to keeping it running. This extended initially from the server level up to the application, and later more operational to application after we outsourced the hardware. The term “tech” or “techie” or “technologist” was a pejorative, more often applied as such because we as people who had to apply technology also had to conform to its limitations. People don’t like the word “no” quantified by physical limitations (i.e. time, processes, memory, bandwidth).

No one in a modern company, certainly not a media company should ever fear technology or not understand both its limits and capabilities. Technology never belongs as the spoke of an organization. It needs to be the hub on which everything else is built.

If your company relies on technology, everyone is a tech person. The music business has always relied on technology and often was a partner in it (see Phillips, RCA, etc) up until Napster. It is disingenuous to be a music or media company and think you aren’t a technology company.

2. To be a partner means being a player

Technology partnerships should exist not through term sheets and “deals” which carry the political weight of business development and legal careers, but through API keys, loose bindings and open exchanges of ideas from machine to machine, and person to person. I don’t think this needs explanation other than pointing you toward IMeem.

Partnerships need to work on both sides, and be as equally easy to create as to sever. When it comes to pass that doing business with Amazon is easier than with my own company, it can’t be any easier for a third party to do the same. I said at a conference that you can’t make an API for lawyers. Well, you should be able to. Certainly I can spin up thousands of dollars of instances on EC2, setup a shop, make a fortune and sell a company all without ever having to talk to Werner at Amazon, or his legal team.

If you want to be a partner in technology you have to be a player in technology. And conversely, if you want to be a partner with media, be a player with it. The only way to equalize this on both sides is to keep the people out of it and let the machines do the talking.

3. When technology is used as a political tool, and especially as a political weapon, no one wins

Technology choices often extend much further than what is right and proper in terms of the actual technology. I’d wager to guess that every technology decision, whether made by a technologist or a business development person (or a combination thereof) is colored with personal political ramifications, cronyism, and engendered bias. This coupled with how important technology is to the core of running a company, and how important the right decision can be to one’s career creates situations in which technology is made political.

This is not good.

In the end, no one wins when technology is used as a weapon in a political war. Those whom the technology is supposed to serve suffer because more effort is put into justification than innovation. The providers of the technology (whether internal or external) loose because judgements about relative merit are all colored with political subjectivity. You end up with two sides, neither getting innovated fast enough, and only innovating in an arms race against each other, not in a customer focused way.

Fixing this is a systemic issue that comes from the CEO on down. As a CTO you can fix this by not allowing it to ever get to this point. Technology at some point needs one voice with authority, as well as systems that allow options to play out in a customer focused way rather than an internally political way (see 5, 7 and 9).

4. Beware of groundhog day platforms

Bad partner decisions can lead to persistent sunk costs; in which the application of money toward implementation once does little to mitigate subsequent integration and implementation costs. If you find yourself spending anything but run rate money six months into a deal it is time to figure out what exactly you are spending on. Expertise should be inversely proportional to money spent in the long term.

5. Software as Service is a fallacy

The fallacy of software as a service is that it means no implementation costs and only operational costs. There is ALWAYS implementation costs, and you need to choose a SAAS vendor thinking of not the operational cost (you might never reach that state, and it’s always the “attractive” cost), but the integration/implementation cost and the disintegration/deimplementation cost. Think in terms of man hours/days/weeks/months.

If that cost exceeds the savings of the best case operational cost, it’s time to investigate other alternatives:

1) Open source + integration
2) Open source + SAAS (reduced risk, better ejection-seat for reduced disintegration)
3) Open source + custom development
4) Custom development

Rarely is 4 ever done from the ground up. The most common method is 3 and 2. You find 2 in, Acquia Gardens, etc.

6. Beware of misalignment of objectives, especially on a technology and business level

If you run your core business on someone elses infrastructure and systems and their core business is in no way aligned with yours, don’t blame that vendor when things go south. You should have always assumed they would as there is no “best case” scenario and there never has been. In the end, it is a perpetual friction between their business and yours; a push and pull in which no one wins. Does anyone remember Atex? I’m sure a lot of former newspaper employees do.

Loosely bound implementations where the risk doesn’t have to be assumed on either side are the best.

7. Stay nimble

A constant in technology is change. I spend about half my time as a CTO type entity researching what is next while also finding better ways to optimize what is present. There are always ways to keep costs low. Always ways to keep your technology stack more future-proof. Being nimble implies the ability to have low risk shallow investments toward new technologies, and allows you to taste before buying rather than having to buy outright.

I’ve seen, in typical IT based organizations technology choices take years in terms of due diligence, implementation and eventual de-implementation. It shouldn’t be this way. A proper stack in which dependencies are mitigated through standards (i.e., message queues, JSON based serialization, Thrift based RPC, etc) allows experimentation without too much pain.

A huge problem in entrenched technology organizations is the pain of change being more than the pain of the current situation. When it becomes easier to work around the limitations of old or inferior technology than to invest in innovation, you have not been building a nimble stack. You’ve built a prison.

8. Reinvent and reexamine decisions constantly

WBR was heavily wedded to Drupal. We launched well over 220 websites in the time I was there. However, not a week went by when I wasn’t constantly second guessing the decision to use Drupal. I would often, as a point of exercise build a mock artist site in WordPress, Expression Engine, Ning or any of the countless other tools and frameworks that popped up over time. A lot of them no longer exist, which proved the validity of our decision to put investment in Drupal, but some have come into their own as artist site platforms (i.e. WordPress and Tumblr) and are starting to be used since I’ve left.

Related to point number 3: software should never be religion. This applies to why you chose something that in your judgement is superior to others who might have chosen differently. The nature of technology means that there are always a dozen ways of doing any given one thing, just like programming in Ruby.

9. Stay curious

I was recently talking to someone that was employed as a technologist but confessed he spends most of his day in e-mail. That made me sad. Curiosity is what drove most of us toward computers in the first place: the ability to manipulate data, systems, graphics and numbers to our very whim played directly with the god-creation desires that also drove us geeks toward Lego, SimCity and Civilization. To have curiosity subjugated by practicality leads toward a stasis that runs counter to the very core of what technology represents.

Often wanton curiosity might look like a lack of attention and focus, but that is furthest from the truth. Focus is only driven by the elimination of that which can distract. An informed decision on what not to do only comes through examination, trial, error and the lack of concern for failure.

The best way I can put it: if I followed the day by day trends from Hackernews in terms of platforms, languages and technology, I’d get nothing done. If I don’t follow the day to day trends from Hackernews, I’m better off not doing anything in the first place. The balance between the two is informed curiosity and is what keeps you innovative in the long run.

10. Embrace anarchy

Chaos has always been my guiding principle when it comes to managing technology teams. I enjoy the disagreements, the entrenched beliefs and more than anything, people passionate about what they feel and believe is truly right. I especially enjoy it when they are divergent from each other.

The great thing about technology is it’s an equalizer. At some point, no matter how convoluted the closures, how Hungarian the syntax, or how esoteric the language, everything will end up getting reduced some way, some how into assembly for execution. Homogeneity does not belong in technology in any way. That extends from the desks employees choose, the computers they want to work on, the way they arrange their monitors or their development environments.

And it also extends to the choices made as a group. If someone wants to prove why something should be done a particular way, they should. I’d fund it and encourage it. So long as there is momentum in the noise and the anarchy isn’t bound by inertia, progress will always happen. And usually the most interesting stuff is the methods that prompt the “WTF?” reaction.

And there you have it. Any other principles?

Update: As pointed out by former colleague, I should point out that we never conformed 100% to what was outlined here. The platform we built was a mess and chaotic in the best and worse sense, caused by all the things that befall technology: lack of resources primarily. The point was lessons learned, not lessons necessarily implemented. Five years is a long time to make a lot of mistakes and a lot of successes.

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.

2 Responses

  1. Well written. This sums up what takes 10+ years of successes, failures, headaches, victories, expenditures, and launches to learn. Really informative and touches on the tech climate at companies in almost every sector these days.

    Another principle I strive to communicate is that technology is not the real foundation of solving a problem. A functioning stack does not always create the right platform. Thorough assessment of core needs and desired/attainable results is often overlooked and tech is used as the band-aid, usually based on individual agendas or poor guidance. Technology is the multiplier – if properly used to address real needs, it can multiply efficiency and effectiveness. And if used wrong or pointed the wrong direction, it can multiply dysfunction and division. But until an honest discussion surrounding real needs happens, horribly misinformed tech decisions will continue..

  2. Clyde Smith says:

    I really appreciate posts like this and I hope folks are talking with you about it outside of what I can see here. A bunch of tweets and likes just doesn’t seem adequate to what you put out.

    I’ve only had a little experience in corporate settings. I worked some on a mobile project as a content person that eventually got canceled. But, along the way, it was the tech people that really held things back. The main guy I dealt with lied to me directly about a key technical point, admitted to it in the meeting and then blew it off. Nobody seemed to care but me!

    Later, I found out that the programmers for this company had convinced them not to create iPhone apps because of some legal details in the iOS SDK which is kind of funny. The lawyers should have been stopping them, not the programmers!

    I think it was really about them not wanting to have to learn to do new things combined with the rest of the people not knowing enough about tech to hold them accountable. They were part of a very dysfunctional machine.

    In any case, I see frustrations from all sides of these discussions. And that’s why, if I had a bunch of money and a music company, I’d be trying to hire or partner with people like you. You’ve always put out so much interesting information that you didn’t really have to share and convinced me that you really care about the team, the product and the customer.

    So I guess this has become a fan letter. Where’s that like button when you need it!