Distilling the Command Pattern with micro-services

One of my favorite design patterns while developing software is the use of the command pattern. I’ve found that varying the pattern’s implementation to focus on a few of the design principles has been a very beneficial exercise. I will start by discussing classic version of the command design pattern, then we'll see if we can get some of the benefits without all of the work.

The formal past

In the past, I've used the command pattern in a more formal heavyweight way. Living in a CQRS world, a request comes across from a client or another piece of the domain that will have the details needed to construct the command object. The command object is often times a class that is just a databag loaded with all of the details needed in order for the execution of the command to take place. When used this way, you can sling your commands to something that can manage your application load. Sling a command onto a queue, give that command a certain priority, and walk away. Async magic happens, Bob’s your uncle, and your work will get completed eventually.

There are some great benefits to that level of formalization of that pattern, such as:

• the ability to “undo” the command
• the ability to record the commands, for future replaying and/or simulation of system load
• the ability to distribute the load of the commands across multiple workers.

Is that formalization needed for micro-services?

Sometimes all the formalization isn't needed. That level of hand waviness isn't always required though, especially when it comes to mico-services. One way to leverage this pattern in a more lightweight manner is to separate the orchestration and "readying the command" from the actual execution of the command. This separation is the secret sauce, the holy grail, and all that’s really needed to start. Micro-services don't’ always need the level of formalization that the classic implementation of the command patterns calls for. The key is decoupling the actual execution of the command from the readying of command execution.

When this line in the sand in drawn, it creates a separation that I've found to be extremely valuable. Here’s why:

Forces design of dependencies up front

When a command executes, it needs to be self-containing. At a bare minimum, it needs to have everything it needs to know how to spin up its dependencies. I've worked on a few commands that require a database connection. Assuming that you didn't serialize the command and pass it along to something else to do that work, I've liked the pattern of passing it a pointer to a database connection or handle to the service layer. With single responsibility in mind, the command shouldn't have to know how it needs to go about getting a connection. This keeps your commands focused and small. This same analogy can be applied to many of the command’s dependencies.

Tell, don’t ask.

By leveraging the command pattern, it's delegating the responsibility that the action that must happen to the command itself. The caller or invoker of the command shouldn't worry about the state of the system, what data may look like, or whether or not the command is a valid action. This logic is the responsibility of the object that is being called, not the caller. By pushing this logic into the command it eliminates the back and forth between the caller and the command.

Dependency injection

When the dependencies are defined up front, dependency injection can be leveraged. A different service, database, or endpoint can be injected into the command without any (or much) rework.

Great entry point for tests

Since out command consumes its dependencies, it also creates a fantastic point of entry for tests. Commands now be easily tested in isolation. Passing in test or mock databases is easy now! This small point alone is enough to sell me on continuing it into the future.

It’s ready to scale

If needed, you are setup to mature the command into something bigger. Because the commands are isolated, how the commands are called is still able to change. You could morph your system into a the larger more formal sense of the command pattern if you’d like to take advantage of some of the things listed above.

As Donald Knuth says in the book The Art of Computer Programming -

     “Premature optimization is the root of all evil (or at least most of it) in programming.”

I've typically pushed for creating the command without worrying about performance at the same time.  Because the command is decoupled from the orchestration of how the command is executed, it’s now easy(ish) to add concurrency or load distribution. When this is saved until the end of the command creation, you can see if scaling is even needed. You may be pleasantly surprised to find that your application is keeping up with the data load just fine, without the added complexity of concurrency.

Scrum, kanban, agile or XP? Which is right for my team?

Design patterns in software development are often times used to quickly describe a set of design principles that when combined solve a certain type of programming problem can provide a solution to the problem in limited number of words.  Each design pattern has it’s strengths and weaknesses to solve certain types of problems. It takes practicing building applications and learning from your mistakes to help you recognize when to apply each pattern.  

 I feel like same analogy can be made for agile processes.  There are certain modes of development process can be applied to help collaborate in the ways that best suit them.  To make these decisions about which methodologies to choose consider these factors:

Get your Mind right

Before you get deep into choosing the processes, reviewing the principles of the agile manifesto helps get my mind pointed in the right direction to remind myself why the process around software development is important.  

Here’s a few of my favorite principles that helped guide and agile manifesto
"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."
“The best architectures, requirements, and designs emerge from self-organizing teams.”
"Continuous attention to technical excellence and good design enhances agility."

Reviewing the entire list is a good thing.  Finding the principles that your team values most will go a long way.  

Get familiar with agile processes & patterns

Extreme Programming
Scrum
Kanban
Continuous Delivery

In order to decide whether or not to apply design patterns to a certain problem, you must understand the problem you are attempting to solve, and understand the design pattern.  If the two are a good match, solving this problem will have a higher likelihood for success.  The same mentality can be applied to choosing agile processes and principles.  Groking the processes and patterns will arm you with the information needed to size up your process decisions.

Know thy self

When deciding which agile processes to apply, one must understand the problem they are attempting to solve.  Many people don’t have the luxury of knowing how their teammates prior to working with them.  Some people are resistant to change.  This resistance may be because what they are doing now is “working” and they want to focus on the actual coding aspects.  When I’m neck deep working on something I’m not thinking about the stuff AROUND software.  

Software is a team sport.  There are plenty of lone rangers and duct tape programmers out there. They don't’ dare talk about process and certainly would not not formalize the manner in which they operate.  I’m not even saying that’s a bad thing either!  This could be a good thing if they are indeed informally following the principles of the agile manifesto.   A well tuned mature team doesn't need to be bound with tons of process.  They inertly grok how to get software out the door.  

For example, here’s a snapshot of how my team at work mainly collaborates:

• Pair programming on demand, not required.
• Bouncer role - keeps the rest of the team focused.
• BDD almost all new code
• Stand ups
• Retros
• Kanban Board
• Another tracker - for larger epic level stories, also keeps kanban board concise.
• Continuous Delivery.  Automate and script as much as possible.  Release as often as possible.

So if I add up the components for how we work.. which model does that put us in?  We’re not exactly kanban, because kanban doesn’t say anything about a pair programming and many of the other ingredients.  In fact neither does SCRUM!  This model doesn't fit anywhere on any existing best practices.  This is any okay thing.

Get ready to learn and grow

“Planning is a process of choosing among those many options. If we do not choose to plan, then we choose to have others plan for us.”   If you choose nothing else, please plan to learn and grow.  Don’t change any of your processes, but do have a weekly or biweekly retro about how the team is working together.  Figure out what isn’t working.. and change one little thing at a time.  Massive process overhaul isn’t necessary if you can always slowly improve.  Fine tuning how you work is especially important for when you get into crunch time.  I’ve found that during crunch times, many want to revamp their processes to align with GTD.  If your process has already been finely tuned to optimize sustainable performance, making any transitions necessary to achieve higher performance is a matter of turning a couple knobs, not trying something completely new.

I really like this mantra:  

Sometimes you just need to start with one thing.

And I think that thing is culture.  Fostering an environment where it is okay to fail, learn and grow is one of the most commonly overlooked crucial components in software development.  The most important thing is that we are always taking a step in the right direction.  When it comes to software development, sometimes taking a big leap can be risky to the big project you are working on.  It’s important to minimize that risk as early as possible.  Focusing on continuous delivery is a model that can help risk more risk sooner. Getting your software in front of your customers ASAP will help you learn if you are marching in the right direction.

So which one?  Scrum, kanban, agile or XP?

I've heard the phrase that “scrum is like training wheels” for agile.  It provides a few of the necessarily components to solving the problem and sets you off in hopefully the right direction.  Once your team matures and groks scrum you can remove the training wheels and modify your processes for what works best for you. It’s important to find the right balance of process to getting things done. The key is growing a software development methodology with culture seeded deep into it’s roots.  This is how great teams and software are built.

Reflections of CodeRetreat 14.05

This weekend, I attended CodeRetreat 14.05.  This is the 4th code retreat that I have attended.  The pattern continued that despite attending these events multiple times, I have still walked away with some valuable lessons learned.  

Here's a few of my key take-aways from this code retreat:

Rust - the language

I had the opportunity to hack around in the rust programming language.  Despite not making tons of progress with the game of life kata during this session, I was still able to experience hacking around with another fun languages.  I enjoyed seeing Rust’s approach to mutability. In Rust, every field is non-mutable unless you expressly make it such.  That seems like a good practice, because it forces you to think about which data you are okay with being mutated.  No surprises with your data changing from underneath you, and that’s a feels like a good thing.

Rust - my skills

I’ve been programming almost exclusively in Go for the past few months.  In one pairing session we began to write some unit tests in Ruby and I mixed up the syntax for GinkGo and RSpec.. a lot!  This was a friendly reminder that it’s important to practice and stay fresh in all of the languages you care about.  Many times when I am writing ruby specs I have plenty of examples laying around in the code base. When it’s a blank slate there is nothing created yet. You are left with only your your memory and the internet for googling examples. Once I got a few rspec tests written it was clear sailing, but there was a humbling moment or two getting the first test wired up.

I also had the opportunity to do this kata in C# with NUnit.  This technology set used to be my wheelhouse, and I was pleased to see that it didn’t take me much time to get back into the saddle.  My fingers even remembered most of the ReSharper shortcuts I know and love. Another neat thing is that I got to try out VIM-VS. I could leverage all of the shortcuts and keyboard navigation goodness that I have learned in VIM, while doing it inside of VisualStudio. This was a new and fun way to experience writing C# code again, but with some added editing niceties I've been practicing. This setup wasn't flawless, and I think this setup would take some tweaking in order for it to be really effective.

Golang

Last time I attended this event, I felt like my skills on Go were not strong enough that I could effectively lead a pairing session with someone who has never programmed in go, so I didn't even push the issue. I spent most of the day in Ruby that day. This time, I was determined to solve the kata in golang this session, even if I was the only Gopher there. Luckily, I wasn't. I had the pleasure of pairing with another Gopher at this code retreat and was lucky enough to be able to introduce them into using GinkGo. I'm glad to see a few others in this area using Golang.

Primitive Obsession

Like many people in this industry, there has been more than once in my career where I have over engineered a problem.  By practicing TDD so much over the past few years, I think that forcing myself to write tests first has eliminated many of the pitfalls of doing a bunch of design work first.  The problem with doing this is I think I stepped out of the “over engineering” category and into the “primitive obsession” category.  Having the challenge of doing Conways Game of Life without the use of any native datatypes exposed was a good way for me to let design for your objects to occur naturally.  I found it also drastically increased the readability of my code.  I plan on keeping a more watchful eye when using primitives to represent anything in the domain.

Remote pair programming

Where I work, we do a decent bit of pair programming.  We also gear ourselves up the right tools to do this job effectively.  We use tmux, google hangouts, and vim daily even when we're not pairing. It’s pretty typically that when pairing remotely that both developers will SSH into one of the pairing sessions.  I learned about a tools call wemux.  This seems to take some of the initial setup work out of the equation and make it easier for attaching to eachother’s tmux session without having to SSH into a shared computer.  

Learning and Teaching

Events like these are always a good experience for me.  Even when I feel like I'm ahead of the curve on some of the exercises like red/green/refactor, frequent check ins, and ping pong pairing, Jim Hurne is still an excellent facilitator and mixes up the exercises enough that can stretch my skills.   When I was in college I got a job as as tutor.  I was recently reminded how much fun it can be to teach.  I was also reminded how much you can learn by teaching.  Being able to justify what you are doing and and why you are doing it keeps your mind right.  I've been impressed how many college students and hobby programmers have been attending the code retreats recently.  It's good to see such a wiliness to learn and grow from an early age in your software development career.  It's fun and exciting to showing a disbeliever how to really write the tests first, or seeing someones mind blown when you show them a new trick, shortcut, or tool.  

Ruby Newbie Guide to Getting Started with Ubuntu for Software Development

This is a reference guide for how to get started using Ubuntu as a development environment.  I use OS X at work and I use Windows 8.1 at home on my beefy gaming desktop.  I wanted to find an ideal development environment at home to make developing a smooth experience.  The languages I currently use most are Ruby and Golang.  This makes using Windows less than ideal for development.  I know there are some tools and support for getting this done, but without getting into a holy war, I humbly choose Ubuntu instead.  To make good use of my gaming PC hardware, I decided to dual boot it and share a set of instructions and tips that helped me along the way on this journey.

Ubuntu Setup

Download the Ubuntu 14.04 image and burn it to a DVD or some other bootable media.

Prepare the drive

Since I was starting with a Windows 8 computer, I created a bootable Partition Magic disc and partitioned the hard drive in half.  I then formatted the new half of the drive.

Install Ubuntu!

Reboot and use the bootable Ubuntu disc your made earlier and install to the freshly formatted partition.   When installing Ubuntu I chose to use the EXT4 file system.

Get your dev environment setup

You can run this script as sudo to install some of the basics like git, curl and vim.  

Code & Text Editor

VIM is my linux/osx editor of choice.  But if you haven't learned VIM yet, consider using an editor that has a lower learning curve like Sublime Text for now.  We'll revisit the topic of using vim in a later blog post.

Setup ssh keys

Hopefully you have a backup of your private .ssh key.  You’ll want to copy that file to either ~/.ssh/keys or a subfolder inside of that directory.  When they are in the base folder every key gets added by default by some kind of ssh keyring manager.  I’m a bit more interested in choosing which keys I’d like to add and when.  So I keep mine in ~/.ssh/keys/users

Bash profile

Setup your bash profile however you like it best. I have a list of aliases I like to have available, and also making sure that VI mode is on.

Virtualization

Many of the applications my team develops now use Vagrant because we want to develop in the same environment that we deploy and to eliminate questions about tool chains to get the application built.  Skip these steps if you don't plan on using Vagrant or Virtual Machines.

Download Virtualbox and install.  
Download Vagrant and install. Vagrant Up's getting started guides are very good if you haven’t used vagrant before.

Make running spinning up vagrant boxes with vagrant actually work.  
There is currently a bug provisioning custom vagrant boxes for the first time with the latest versions described in this guide.  This Ubuntu First VM dance link will fix you up.

If you need Cisco VPN

Run this huge Cisco VPN setup script.  This is a HUGE .sh file.  Looks like it contains enough binary in it to produce their client from scratch.  This should be posted somewhere public on the Cisco site.. but I sometimes have a hard time finding it.  Use the link above for one I know works.

Another option that many linux users may prefer is using OpenConnect with Gnome's network manager (via network-manager-openconnect-gnome).

If you need Oracle SQL Developer

Then first off… I’m sorry.  It’s a pain in the neck to get installed, but worth it.  At least it was when I did this with Linux Mint.  It looks easier to do on Ubuntu, you could try following this oracle noob guide.

Making Windows and Ubuntu play nicely together

Download Virtualbox for windows

Download an Ubuntu image to be used as a shared VM between OSes.  If you download this while on Windows..make sure to just run the .ova file.  You can’t mount or create the VM from within the VirtualBox manager.  The ova file is a package that has everything you need by just running it.  You do also need to download Oracle VM Extension Pack if you want USB 2.0 support.

Credentials for Ubuntu base virtual image -  ubuntu/reverse  You may consider changing these later.

Enable file sharing between the Windows 8.1 and Ubuntu so you can share your VMs and use them anywhere.

Closing Notes

Thanks to @jesse_szwedko for reviewing this article and providing some suggestions.

Golang for the Rubyist

The following blog post is a reference post to a lightning talk I'm performing tonight, Thursday, March 6th at PghRB - Pittsburgh's Ruby Users Group.  

You can find a copy of my slides here, and a copy of the source code here. 

Top 10 things about software development I wish I would have learned in college

Recently a colleague of mine, Rachel Thompson, posted this on twitter:

This inspired me to gather up some thoughts about what things I wish I would have known while still in college.  If I were going to hop in my Delorean and go back in time to drop a few tips about software development to my younger self here's my top 10 tips I would give myself.

1.) Source control

I wish that I would have learned how to use source control earlier.  I truly can't believe that something as fundamental as source control is missing out of the traditional curriculum of a software developer.  It really doesn't matter which one you learn.  This will help you immediately don't wait until you graduate.  Checking in stable versions of code and documents is so valuable.  There will be times where you wish you could go back to the last point where your program was working the way you wanted it to.  Source control is a safety net. learn how to use it.  

2.) Users groups & conferences

Networking is important, start doing it now.  Users groups are a great way to introduce yourself to other people who are interested in learning and growing.  I've met some of my favorite software developers at users groups and conferences.  It’s a wonderful thing to find people who are passionate about their craft.  Consider reading the passionate programmer, run with the "A crowd", surrounding yourself with people who are excellent at their job.  It's only a matter of time before their good habits will wear off on you. 

Imagine the opposite.  Imagine that you accidentally found yourself at a really bad software development shop.  You are surrounded with people who are there to punch a time clock and collect a paycheck.  They may browse the internet more than they write software.  This sometimes actually happens.  There are some large corporations where bad developers can slip between the cracks for multiple years.  Don't their let bad habits rub off on you.  Users groups and conferences are a great way of meeting people who really care about their craft.  It's a good opportunity to keep your finger on the pulse of where the talent in your area is working.  Surround yourself with successful people and you will be successful.  

3.) Technologies learned

When I was an intern, I worked for a cable company that provided the software that would be on your cable boxes.  We wrote the software that would show you On Demand content, pay per view, and allow you to record TV shows.  The language they used is called VODScript (Video on Demand script).  Luckily, I had the good sense not to get myself tied up with a proprietary language.  Make sure that you don't get yourself mixed up in a language like that.  You don't want to pigeon hole yourself at this point of your career.

4.) Empathy

I've read somewhere that more projects fail because of people problems than technological problems.  A group of skilled engineers can typically solve most design and engineering problems.  Often times teams start to struggle when people problems start to surface themselves.  Someone once told me that they'd "rather dig a ditch with friends, than work on a great project with a bunch of assholes".  Although I don't necessarily enjoy digging ditches or physical labor, I do agree with what this quote is trying to get at. 

Respect your fellow engineer.  People you work with are only really willing to discuss things that are socially acceptable to talk about.  If you can have a safe environment to state breaking down some of those barriers, your team will grow.  You will learn to understand each other.  It's important to feel close to those that are on your team.  Odds are you will spend more time directly interacting with them than you will your significant other.  Love the people you work with.  Love what you do.  Help people share those beliefs, and make them feel safe in confiding in you and your team mates.  You cannot be successful without your team being successful.

5.) Being humble & maintaing healthy relationships

When I first started my career, I think that one of my primary objectives was to show my boss how much of a rock star that I am.  I later found that I prefer to have a coaching relationship that I have with my supervisor.  No one is perfect.  Be honest with your supervisor.  Tell them where you'd like to grow.  Ask them for their honest feedback.  It's a much healthier relationship to have a mentor/coach that you report to than trying to convince someone that you are skilled.

6.) Tech is a tool & not necessarily the hard part

If you'd read this blog much, you've known that within the last 6 months I made a semi-drastic career change.  I left my very hardware centric, .NET development shop to a Ruby/Go web shop.  I really envisioned that the technology would be the hardest part of this transition.  It's not.  The hardest part of this transition is the domain knowledge.  You can google "How do I do X in Rail" or "Best way to blah in .NET", but you cannot google for product specific things.  When you start a new job, take time to understand the domain as much as possible.  Technology is the easy part.  Internal training is expensive and time consuming.  Don't make your co-workers repeat themselves.  

7.) Being a software tester

Being a software tester (QA) is a great intro into the software development career path.   I started my career as a QA.  I interned as a QA, and spend approximately 6 months as a QA in my first professional job.  The line between a software developer and a technical skilled QA is blurring.  I'm to believe that the salaries are also similarly blurry.  If you more passionate about finding bugs in other people’s software and general breaking stuff being a QA might be a good fit for you.

8.) Practice

One of best way to become a skilled software developer is by practicing your craft.  Have you heard of Code Katas yet?  If not, it's a great way to independently improve your development skills with fun problems that are just challenging enough to stretch your.  Using katas is also a good way to learn a new language.

9.) Time management, staying focused and consuming information on demand

E-mail - Turn off e-mail notifications, they are distracting.  E-mail is typically seen as an urgent, but not important activity.  Turning off chat/e-mail is a good way to help you stay focused when you need to be productive.  Also, consider managing your e-mail with via Inbox Zero.

Blogs - Find some interesting software development blogs, and subscribe to them in an RSS feed reader.  This is a good way to consume the media that you are interested "on demand", and not necessarily whatever is trending on twitter or Facebook.  Try to not have too many blogs in your feed.  It can be intimidating when you open your feed reader and you have 100+ things to read.

Timing management - I love using the pomodoro technique while coding or performing any task that I need to stay focused on.  Not only will it help you stay focused for a small chunk of time, it's also a good way to step away from the problems that you are trying to solve.  It's surprising how many times you will come back into a programming session with a different perspective because you temporarily separated yourself from the problem.  

10.) Money Management

You spend 4-ish years in college.  There is a demand for developers out there in the world.  There is a good chance that when you graduate from college, you will be able to find a job.  Assuming that you can land your first job, you will also land your first professional paycheck.  Now what are you supposed to do with it?  What's the smart thing to do with it?  Someone once told me that it's important to spend at least 10% of the time that you do studying for your career, to spend studying financial management.  Don't make dumb decisions with your money.  You've studied long and hard for this pay off.  You will work long hours for it.  Educate yourself on financial independence, investing, and debt management.  I guarantee it's worth it.

Simple Made Complex

What a crash course my learning experience has been over the past few months.  I must admit, my original intentions with this blog were to post frequently but I have been lacking in that to say the least.  I imagine this patterns is consistent with others who have thought. “Hey, I’m going to start a blog about that!” but became much more engulfed in what they were learning or experiencing than the desire to blog about what they were experiencing.  That said, I do have quite a few interesting things that I think are worth sharing, and I hope to start posting about these topics.

Out of everything that has been in a state of flux the absolute biggest change has been… *drum roll please* working primarily in a terminal based environment.  Prior to this job, I'd guess I'd spend less than 5% of my time in some kind of terminal..mainly Windows PoweShell when I needed it.   Now, I'd say that I spend over 80% of my time inside a terminal window.  My team uses VIM as their primary IDE, and tmux for switching between several programs in on terminal .  VIM is a text editor that is built into most UNIX systems.  It’s notorious for having a step learning curve but after you've overcome the learning curve many swear by it.  Here's an example of me hitting the learning curve pretty hard with VIM.

Simple made complex

1. Now in VIM.. say you want to copy and paste 2 lines of code.  Here’s how you go about doing it.
2. Ensure that you aren't in insert mode 
3. Ensure your cursor is on the spot where you'd like to begin selecting the text you'd like to copy.
4. Hit V to begin character based visual selection.
5. Move your cursor to the end of the text to be copy.
6. Hit Y to copy the text.
7. Move your cursor to where you’d like to paste text
8. Hit P to paste.

Blamo! Copy paste via vim.  That’s not so hard, right?  Well.. I guess not.  Seemed a bit laborious to learn another way to copy paste.  But in all fairness, VI has been around longer than I have.  Now I need to remember CTRL + C /CTRL +V for windows, Command C/Command V for OSX, and the dance listed above for VIM.

Now let’s say that I need to take the code that i just copied and put it into a bug report.  Unfortunately, by default, VIM doesn't share the the operating system’s buffer… so you need to figure out a way to get that text into a buffer that’s accessible outside that VIM session. You can do things like using the * register to reference the system clipboard, but that doesn't work in every Unix OS.  You can set clipboard=unnamedplus in your .vimrc file but it requires at least vim-7.3.74.  If you are on OSX you can use the command line tool PBCopy.   And...there seems to be 10 more ways to solve this problem.  There are times when I’m really impressed with the things I can do with VIM, and there are other times when it comes to things like this where I grow frustrated that things as simple as copy/paste can be this complicated.  It seems very silly that I actually need to go to google and search for "how to copy paste VIM OSX"  ... but I digress.

You can get by in VIM by using the most basic commands, but it’s real power comes from learning all of the tricks in and tips, installing custom plug-ins, shelling out to the command line within VIM.  Here’s a link to the "best” VIM tips.  I've found reading these to be very helpful, and I've tried to teach myself at least one new trick in VIM everyday.  Once you get beyond these initial learning/growing pains there are some really wonderful things that you can do with VIM.  There are countless tricks in here that i know I couldn't do with my last few IDEs.  The trick is continuing to learn a new thing or two everyday to make that initial learning curve worth it.