A couple weeks ago at Networking Field Day 9, Brocade presented with their usual A-list of networking gurus. One of the presenters was Jon Hudson, a very engaging, visionary speaker. His talk, shown below, was about the state of network programmability.
During the conversation (which is well worth watching), discussion turned to the question of “will network engineers become programmers?” posed by John Herbert of MovingPackets.net. Jon Hudson’s response elicited applause from the room. He said:
“The trouble I have with that statement is, most network engineers I know, like myself, we know how how to code. We went to school for it, and we chose not to.” – Jon Hudson
The conversation went on to discuss the value of programmability for the sake of consistency in the management and configuration of large-scale network fabrics (which I don’t think anyone would really debate as a “Good Thing”), but Jon’s quote about being a programmer and some of the sidebar that flowed from it created a fair bit of activity in the Twitter stream. Following the presentation, my attention was called to a mailing list on which a question was asked about networking engineerings being “given a pass” on the need to “code” while other IT disciplines have been forced to embrace it. The remainder of this post is a slightly-edited (for context) copy of my reply with my own thoughts on the matter:
Jon’s quote resonated with me because, well, it is me. I have my CompSci degree from a (relatively) prestigious engineering university
, but along my journey to earn it, I discovered the distinction between what I consider “tools” or “scripts” and “software.” Scripts were the little programs I wrote for the first few years of my training. “Calculate the leap years for the next 3 centuries”, or “take this file, iterate through the entries on each line and munge them in some way.” That is fun, useful, mentally stimulating, and easy to replicate. They solve a specific problem and can easily be replicated and tweaked to solve a slightly different problem. Most of those programs are also fragile, inefficient, probably insecure, etc. My experience with writing real software was of the “Your function will be given as input a pointer to a vector containing values which must be validated against the established data dictionary. Any exceptions thrown during execution of your routine must be handled. Your routine must run in O(n*log(n)) and produce as output a pointer which shall be passed back to the calling function which references a new vector containing the calculated values available only to the namespace of the calling function. Document your code thoroughly and demonstrate that your solution was accomplished with the minimal number of memory allocation calls” variety. That’s about when my interest in “developing software” died.
The long and short is that I don’t think networking admins should be given a pass from having to work efficiently and build tools if necessary. However, I consider scripting and tool building to be very different from serious “coding” or “being a programmer.” And that, I think, is a point on which some people get lost. I realize that some understand that distinction, but I think a lot of the “Code or Die!” rhetoric is less specific about it. I also think a vocal group of people (majority? minority? I really don’t know) are trying to cram the concept that “you *must* be writing code if you want to have a future” down our throats. I disagree. We *must* work efficiently and with high quality/low error rates. If that is best done through building some tools to augment your toolbox — great! If you work in a cloud-scale environment or somewhere that has embraced DevOps automation to near-complete levels maybe that does mean everything has to be programmatic. But, you know, sometimes it’s just easier to go tweak the damned firewall rule yourself. And sometimes it’s faster to do that once a week than it is to write a script that does it for you which you have to conceive, POC, develop, debug, and then, most significantly, maintain.
source – xkcd.com
Also, as a consultant, I have seen the effects of systems automation taken to bad extremes. Wizards building intricate clockwork that nobody in their own organization understands, which is quite unfortunate when the wizard finally gets bored and resigns. Then, no one understands how the company’s custom network access control system based on identifying OUIs of MAC addresses specified in DHCP requests actually works. This happens because this stuff gets written like tools. Quick, dirty, often not well (or at all) documented. Those solutions differ significantly from production-level code which has to account for exception handling, input validation, secure coding practices, memory efficiency and garbage collection, QA, regression testing against test plans, controlled release cycles etc., etc.
source – xkcd.com
I’ve talked with some of my close college friends who earned their CompSci and CompSystems degrees right along-side me who went on to build careers now with 15+ years of experience in software engineering. I’ve told them of how the buzz is that “everything must be codified” and all IT people will have to become “programmers” or “developers.” They have a hearty laugh over that every time.
To be sure, in this new world the idea of the NetDev does exist. But honestly I think there are some folks in the Twittersphere who aren’t just embracing “code or die” as an IT admin, but are instead making a career transition to part- or full-time tool builder or even software developer and are confusing that move with being a network or systems admin who must, by definition, “automate all the things” using Python. By all means, build tools when the tools help you get something done faster, better, cheaper. And demand of everyone on the team that things get done better, faster, cheaper. But I don’t believe in “coding” or “developing” as an IT religion. If you need a saw, just use a saw. If you need a jig to saw better, build the jig. Be careful about becoming a jig-builder who forgets how and why to use a saw, unless you are looking for a career change.
I believe the tool-building will happen and has always happened in IT, but will become more prevalent in networking gear as the capability becomes more widespread. I expect development of SDN apps will be done by software development companies, whose product is SDN apps, and will be written by software engineers who know how to properly write applications. Or by open-source developers who are usually still “real” software engineers by trade. The rest of us practicing IT folks should be expected to work efficiently and accurately. Sometimes that suggests or dictates building a tool, and sometimes it just doesn’t.
That’s my two cents. It’s probably wrong.
I’m interested in your thoughts on the matter. Leave them in the comments, even if it’s to remind me how wrong I am.
Brocade was a sponsor of Networking Field Day 9, and as such indirectly helped to pay expenses associated with my attendance. At no time did they ask for, nor where they promised any kind of consideration in the writing of this post. The opinions and analysis provided within are my own and any errors or omissions are mine and mine alone.