Years of toiling away with Java Virtual Machine (JVM) technologies made me realize that the only thing that really matters when developing in the JVM space is that you can intelligently talk about it. These complex projects rarely get deployed to production or used by users, so it’s usually sufficient to just have a headline and be able to talk circles around your businesspeople to confuse them. Then, you can go have a beer.
Written in state of the art Kotlin using complex AI (if/else blocks and switch statements) it is truly a groundbreaking piece of technology.
Please send me copious amounts of money so I can buy a beach house and retire. Thanks!
Shortly after, I came across “Why I Struggle With Node” by Graham Cox. It’s a great post about why he loves Node.js, but gets frustrated with its ecosystem and prefers that of Java.
The post made me think of my conversation with my friend. I linked to Graham’s post as a starting point, added some additional thoughts and sent them to my friend to give him a place to start with Node again. After I wrote it up, I thought it might be useful to other Java developers who haven’t taken the dive into Node.js and would like to, but might be intimidated or not know where to start.
If you’re in that boat, I suggest you first read Graham’s post. It has a great list of tools to start with that translate well into front end code too.
After you read that post, here are my notes and addons that I think might be helpful if you are coming into Node development from Java in the beginning of 2017:
Also, before you dive into creating a full app or start setting up tools, learn the Promise API and use it from the start in your code. It will make your code easier to reason about, and will make your life much more pleasant once you get past the initial learning curve. If you can, use the Bluebird library for Promises. It has lots of great extras on top of the native implementation, and rumor has it that it’s even faster.
Like Graham, I use Grunt for building, too. I haven’t tried Gulp, I’m sure it’s great, but I’ve used Grunt for a while and have found it effective. Though, like Graham, I do miss the fact that Maven takes care of a lot of the gruntwork (ha ha!) for you. For my team’s apps, we have separate Grunt tasks and configurations for building front end and back end code since folder structures are different.
This is mentioned in the post but I thought I’d provide what I’ve used: Istanbul through Grunt. It’s nice and simple and does the job well.
Graham mentions that Node logging isn’t as nice as what he’s used to in Java. That’s true, but I’ve used Winston quite effectively in production apps.
Graham is right, debugging in Node is hard. I attribute some of that to it’s asynchronous nature, though.
I’ve used node-debug effectively as a debugger. It’s quite nice, but a real pain to setup. But, if I’m being honest again, usually I try to use Winston (mentioned above) to put some nice low-level logging statements in place so I can debug with those, then turn logging levels up and down as needed. Sometimes those low level statements help when you’d least expect it.
When I first started with Node, we were using a dependency injection framework in Node since my team was used to using Spring in Java. After a while, we just got frustrated with it and abandoned it.
Graham’s comments about integration testing being much easier in Java are spot on. I haven’t come across nearly as many drop in, in-memory replacements for connected systems like databases or message queues. This makes it tough to integration test. What I’ve done in the past is to create throwaway Continuous Integration database instances at the beginning of the tests and then clear them out at the end of the tests. It’s not that bad once you set it up once, but I yearn for the ease of using something like HSQL in Java.
Team Development Environments and Docker
My team has half Windows users and half Mac users. It makes for a challenging environment for everyone to setup their development environment in Node.
If you’re on a team with developers using multiple platforms, and deploy to Linux, what I recommend is to create a Dockerfile which defines your development environment. Then, use a Docker volume mount to put your source code inside the container. (Side note: Use an official Node.js Docker image as your base image). This lets you use normal Docker build/run commands to bootstrap your dev environment. The volume mount lets you dynamically update your code so you can still iterate fast, as if you were working on your local machine. It works well across platforms (no worrying about exceptions in Windows), and as a bonus, if you ever want to run your app in a Docker container, you’ve already got it more than halfway there. I’m hoping to post more about this in the future.
I’ve been developing in Node for about 3 years now, and I still really love the developer experience. Sure, I’m probably not using the latest and greatest tools that just came out, but this workflow has gotten me to a good place where I can be very productive and crank out some good quality code.
I hope some of these notes help you do the same.
What tools/techniques/libraries did I miss that were important for you when you came to Node from Java?
As a coder, it can be easy to get mired in details. Our job is to manage the details of a business in code. And with the best of intentions, we may even create more details for ourselves through our designs and technology choices.
Worrying about so many little things, it’s easy to lose sight of why we’re putting fingers to the keyboard in the first place. Asking a simple question: “Why?” can help you rise above all the details to see the forest through the trees and make better software.
When assigned a task or project, many times my first reaction is to think of which technology is most appropriate to complete the task, and then quickly after that, start formulating how I would code it.
The technology is the fun part! And it’s our job, right? Well, yes and yes, but immediately jumping into a technical solution is a guaranteed way to make things more complex than they need to be.
And while you’re making things more complex for you and your team, you might be hurting your customers by not delivering what they really need or want.
Next time you get a project, instead of opening up an editor, or Googling, or drawing a diagram, first ask “Why am I solving this problem?”
If you don’t know why, ask the person who gave you the task. If they don’t give you a satisfactory answer: ask more questions. If they still can’t answer your question, and you can feasibly do so without getting fired, ask them who else can tell you why.
“My boss told me to.” is never an acceptable answer on your part or anyone else’s.
What is a good answer? I don’t know, stop asking questions! Just kidding. But, it should probably address one or more of the following:
A specific use case. For example: C-Level executives need this new report to make decisions about budget next year. Or: All users need to be able to save their login credentials in a cookie so they can save time each time they access the app.
Ease of operations. For example: Formatting log messages in XYZ format will allow the application support team to parse them easier and identify causes of bugs in logs quicker and make our customers happier.
Speed or quality of changes. For example: Writing an automated acceptance test suite will help us react better to customer needs by getting features out faster.
But ultimately, a good answer to “Why?” is one that makes sense to you and isn’t simply “It’s my job to write code” or “My boss asked me to.”
It is staggering how a simple “Why?” can halt people in their tracks and cause them to change their decisions, often for the better.
Especially on a team of seasoned people who have been with an organization for a long time, it’s good for someone to keep asking “Why?”. Chances are, there’s a bunch of people in the room with differing opinions, and the question will get them out in the open. Sometimes, it might even turn out that the task shouldn’t be done at all. Which is good, that frees you up focus on more important things. At the very least, you’ll probably dig up some serious “gotchas” about your task in the process.
Your Leaders Are Human
When you’re looking for answers, be patient and remember that the product owner/architect/supervisor/bossman/leader you’re asking is busy and he is human, just like you. He has lots of stuff he needs to get done, and sometimes he make mistakes and has lapses of judgement. Your job is to consult with him to get things done for a business. You can’t consult with him if you don’t fully understand why you’ve been assigned work.
If you don’t understand why, or he didn’t explain: Consider it part of your job to ask more questions. If he is at all a decent leader, he will take the time to explain the value of the bug you’re fixing or the enhancement you’re doing.
If your leader’s answer ends up being “because my boss told me to” and you can’t get any further then, well, it might be time to find a new organization. Unless, of course, you’re up for the task of managing upward, which is probably a good topic for another post!
Reducing Accidental Complexity
Earlier, I mentioned that we sometimes create unintended details through our everyday choices of technology and design. These unintended details are generally referred to as accidental complexity. This is opposed to essential complexity, which is solving the “real problem.”
Accidental complexity is part of our job and we can’t avoid it. But we can mitigate how much of it we create! Asking “Why?” helps.
When you ask why you’re doing something, the answer of “Because technology X needs technology Y” should be an immediate red flag that someone doesn’t fully understand the business problem at hand.
Writing code to solve a problem that other code created, or to solve a shortcoming of the AnguReactBerJS framework is not providing any value to the customers of your software. You are probably just creating more accidental complexity instead of solving a real business problem.
Asking “Why?” will help you focus on the simplest way to solve a real business problem instead of simply fixing technology. You might be able to bring your solution up a level to eliminate a technical issue or shortcoming altogether.
Say you’re given the task to add a set of Automated Acceptance tests for your application. Do you know why you’ve been given that task?
Is it because we want to use SeleneCucumBybara, the latest, greatest testing framework? Probably not. Is it because we want to shield ourselves from creating bugs and allow us to be more flexible with the codebase so we can get features out quicker? Now you’re probably on to something.
Knowing “Why?” will help guide you through the immense pile of decisions you’re going to make when you start writing code. You’re probably going to make multiple decisions per line of code you write. You want your coding decisions to be made with the right values in mind. For instance, if you want to safeguard against a changing codebase, knowing this might help you target tests for the area of code which changes the most.
Knowing Where To Cut Corners
“Cutting Corners” is a phrase engineers hate, myself included. But in the reality of most business software development, a shipped product is better than a perfect product, so there’s going to be some paint dripped on the floor.
Asking “Why?” will help you know where it makes sense to spend less time polishing code and where a cut corner is acceptable.
For instance, if the answer to “Why?” is “We need to get this feature out fast so we have first mover advantage” (meaning time to market is important), maybe it’s more important to have a few solid end to end automated acceptance tests than it is to have good unit test coverage.
Or, if the answer is “This bug is causing customers to be charged $5 per order by mistake”, you probably want to spend more time writing tests for your code and less time polishing the front end.
Remember that the concepts of Technical Debt and Backlogs (in Agile) exist to help us make sense out of the imperfections we might create. These are ways for you to quantify your technical risk, track it, and hopefully address it later.
Does a problem happen often? Ask “Why?”. This will help you figure out if a problem is systemic. Fixing the cause of a systemic issue will save the team repeated effort which will allow everyone to focus on solving problems with more value.
If you’re new to software development, it might seem intimidating to ask such a simple question all the time. You might be afraid of seeming like you’re not good enough. But newbies are in a perfect spot to ask “Why?” A good team should have an understanding that a person new to software won’t know it all and needs some help. They should also come to realize that questions like these will help the whole team get a better understanding of what they’re doing and if it needs to change.
I think it can be harder for someone who has a lot of experience to ask a simple question like “Why?” all the time. With more years of experience come greater expectations on the part of others. This makes it more likely for a seasoned developer to perceive themselves, or possibly even be perceived, as being incompetent, even though they are anything but.
Regardless of your experience, it takes some courage to ask “Why?” Just keep on asking and feel dignified knowing that you’ll find better answers and do better work than those who don’t.
In one of the talks I attended, Jeremy Keith reminded us that though the web has become a complicated beast, it is still amazing in its simplest form. And we shouldn’t forget that.
This sentiment rang very true to me – It made me recall the pure excitement I felt back in the mid-1990’s, when I was 13 years old, putting my first web pages on the Internet. A kid like me could create something from nothing, and better yet, self-publish it and have it accessible by the whole world. Anyone could see my content!
Along this path, I forgot about the power of a browser rendering simple HTML and CSS. And perhaps more importantly, having that content be globally accessible in one simple place – a URL.
These ideas of were underscored by almost every other speaker at An Event Apart. They reminded us that in order to create the most useful web sites for a user, we need to forget about fancy layouts and CSS. Instead we should focus on content first and ask ourselves: “What is most important for the user?” We should make that content simple and easy to find, regardless of how advanced their device or web browser is. Then, get it captured in plain, semantically meaningful HTML markup. The rest will fall into place.
I write this with the hope that it will help me remember these ideas for the next web app I build.
The first part of this post is in the form of a screenplay for an infomercial:
Narrator: “Do you work with computers in any capacity beyond Web Browsing, Microsoft Office and Gaming?”
Black and white footage of a software developer is shown. He is clicking madly and is extremely frustrated about having to rename 20 *.txt files in a directory that contain a bunch of Word documents.
Narrator: “Are you fed up with all that clicking when you need to do some repetitive task on your Mac or PC?”
Developer: “There’s gotta be a better way!”
The developer throws his hands up in frustration and pushes his laptop out a window. Stock footage of an explosion at the bottom of a canyon is shown.
Narrator: “Well don’t go crazy! You can save yourself tons of time by just learning a little bit about our old-fashioned computer friend: the command line!”
Footage of the software developer is now in brilliant color and shows him in front of a beautiful Macbook, with the Terminal open, smiling like a madman.
I started writing this post and it ended up sounding like an infomercial, so I just went with it. When my wife and I get frustrated with some kind of everyday annoyance, we always joke: “If this was an infomercial, we would be doing this in black and white and not color! There’s gotta be a better way!”
Am I rambling? What’s my point? I feel like working with the mouse for serious computer work is like seeing the world in black and white instead of beautiful, magical color.
I am inspired today to write this because, last night I read possibly the best explanation ever of why you need to stop using Graphical User Interfaces and start using the command line in Unix/Linux.
Is this content new? No. It is old, but it is as timeless as anything can be in the computer age. It is written for the average power user of a computer that doesn’t yet use the command line for his work.
I can’t possibly do a better job than Oliver trying to convince you why Linux and the command line is so powerful, so read the first three sections of that Wiki. Come back and let me know what you think.
If you’re not convinced, you should just throw your laptop out the window and stick to your smartphone and maybe find a job in woodworking.
For Linux power users
You’re probably (hopefully?) nodding your head in agreement with everything I said, and maybe didn’t read that article. You might think you know it all, but I would recommend taking a look. It will give you fresh eyes and a fresh mind. It will inspire you and remind you why you love Linux. You might learn something you didn’t already know, or, at the very least, save the link and send it to the next person at work who steadfastly refuses to stop click-click-clicking.
Also, please take a look at Oliver’s list of 100 Useful Unix Commands. You will know a lot of them, but some of them you won’t. And I bet you will learn something about some commands you already thought you knew as well.
My favorite was the “cd_func” under the pushd/popd section which allows you to track and navigate back through your visited directories. Awesome!
Then, I heard about it a lot. A LOT. Developer after developer, blog post after blog post. The book’s central idea is to automate (almost) everything in your software development processes. By doing this, the theory goes, you can deliver value to your software customers faster, with less risk.
There were many potential uses for these ideas coming up in my day-to-day work, so I finally decided to read the book to see what the hype was about. It was definitely a worthwhile read, and as a way to help me remember some key points from this book, I decided to write up this post. I thought it might spark some interest in others to read it as well.
Why read this book?
If you’re involved with software in any way: a developer, a sysadmin or a manager, you really should know this stuff.
Reading it cover to cover may not be necessary, which I’ll explain below, but you should at least be familiar with the concepts and techniques talked about in this book.
Even if you think you know the concepts, you should still read it. Admittedly, some of it is simply becoming standard knowledge and some of the tools it refers to are dated. But, the details in many of the chapters matter and will be really valuable for you.
Before you start reading…
Plan on reading about a chapter or less at a time, then stopping to digest it for a day or so. Because it’s written so well, it’s almost deceiving how dense the material is.
As I alluded to before, don’t feel like you need to read the whole book. This is mentioned in the book’s introduction. There are a few essential parts I’ll outline below, but otherwise, pick and choose as you see fit. The authors specifically mention that each chapter should stand on its own.
Here are the parts I consider must reads:
This sets the stage for why the book was written and the best way to read it (in the author’s opinion).
Chapter 1 – “The Problem of Delivering Software’
This chapter addresses fundamental problems the book sets out to solve and general principles it follows. It also will define certain key terminology that you’ll need to be familiar with throughout the book.
Again, this an especially important read for managers who may have been away from code for a while, or (gasp!) managers who have never coded before. It will help you understand how manual processes are fraught with errors, cause your developers to tear their hair out and generally make them (and by proxy: you) miserable.
Chapter 2 – “Configuration Management”
This was a standout chapter for me. Humble and Farley present why it’s important to be able to automate the creation of your software environments completely. Most of this stuff I was already familiar with, but the way this information is laid out is the best I’ve seen: Step by step, introducing the problems and high-level solutions. It helped me solidify my existing understanding and give more depth to what I already knew.
This is the type of information you need to convince others that automation of your environments is of utmost importance and also tell them that “Yes! It’s possible!”
If you’re less technical, let your eyes glaze over anything that you might not understand. You’ll still come away from the chapter with plenty of new ideas. Just be careful about how you present them to your developers if they’re not already on board :)
Stand out areas
Infrastructure Management (Chapter 11)
This was probably my favorite chapter in the whole book.
For a large, dense textbook, I found myself on the edge of my seat in this chapter. I could not get enough of the ideas that the book presented to solve common issues with software environments getting out of sync and how to prevent them. I lost count of how many times I thought:
“Wow, this would have helped with [insert huge, time draining issue here].”
There are also some important tips in this chapter regarding keeping your environments maintainable. Things like logging, change management and monitoring.
You could almost consider this chapter a crash course in good system administration. And it applies whether you’re running in the cloud, your own datacenter, with one server or thousands.
A key theme throughout the book is that you should use source control. I knew source control was important, but after reading the book, I realize how crucial it is to everything in software. Now, I want to Git all the things!!!
They make it clear they are not fans because feature branches deviate from the main line of code and thus stifle Continuous Integration across developers. I can see their argument, but having used feature branches very effectively, I have to wonder if their opinion has this changed since 2008 when the book was written. Things like pre-flight builds help with these sort of issues.
One of my key takeaways from the book (from Chapter 8) was that to effectively run Automated Acceptance Tests, you should build a test API layer into your application. This layer captures important use cases which you can use as building blocks for real tests. This is lightyears more maintainable that using UI driven tests, which the authors say should be used, but as little as possible.
This was eye opening and a very useful idea I hope to implement someday soon.
Also, they give good guidelines for how to categorize testing and when it makes sense to automate a test and when it doesn’t.
Chapter 12 – “Managing Data” also gives some really great tips on how to manage test data.
My Only Criticism: Not enough advice on where to start
My only criticism of this book was that it preached about the ideal state of Continuous Delivery a lot and didn’t spend enough time on how to get started if you’re in a rut already. Which I’m sure is where most readers are at.
I’m sure I’m not alone in that I desperately want to reach the God-like software state these guys describe, but I have some harsh realities of a large organization I need deal with before I can get there.
What follows is a list of some of the questions the book raised for me, but didn’t answer. My intent here is not to answer these questions, but to highlight some areas know I need to get more information on, and that you might, too.
I would love to hear your comments about any of these:
Differing levels of enthusiasm
If you work in a medium to large sized organization developing software, I’m sure you have to deal with a range of enthusiasm for the job. This ranges from older people who are completely content to run a deployment from a Word document, to overly enthusiastic cowboy developers who do what they want when they want, without the faintest whiff of a plan.
How do you herd these cats and get them drinking the automation Kool-Aid? How do you get people who aren’t excited, excited? And how do you contain the people that want to go crazy shaving yaks without a good, solid vision and plan to get there?
This is something I’ve been thinking about a lot since I read the book and have tried a few things out, but that’s for another post.
With individual contributors moving your digital widgets, a lack of enthusiasm is one thing. But if you have a lack of enthusiasm in management (especially upper management) this can present a serious roadblock to making progress towards these ideas. Or even worse: What if they’re opposed to the costs of implementing automation?
I’m still trying to find good ideas for convincing upper management to back these ideas (Without telling them to read a textbook…). Management wants numbers, and numbers for this stuff are hard to come by. It seems as though storytelling is more effective, but not everybody buys stories…
Furthermore, what about dealing with upper management who pushes too hard on these concepts without really understanding them? You know, the type who reads about DevOps in an in-flight magazine then lands and puts “Must implement DevOps by June” on your yearly objectives…
The book mentions, or alludes to, DevOps a fair bit.
The authors recommend having a unified team with development, operations and QA all working together in harmony. That’s great – at a startup or an organization supporting one large application.
But, what if you work in a large, siloed organization that supports hundreds of medium sized applications? How can you get these ideas to work?
Side note: DevOps is a buzzword I’m beginning to dislike. I appreciate what it’s after, but it seems to have been engulfed in meaninglessness by big companies wanting to make a buck.
How do you manage continuous delivery in a team of people who range from right out of college to senior developer. How do you get newbies started? How do you teach old dogs new tricks?
If you made it here, I hope you’re convinced you should read the book, or at very least add it to your Amazon Wishlist :) If you end up reading it after seeing this post, come on back and tell me what you think.
If you’ve read the book prior to reading this post, I’d love to hear your comments/criticisms/ideas in the comments.
After reading this book, if you’re not convinced that moving towards these concepts is worthwhile, you should probably find another profession.