How to automatically lint your Python code on commit

Increasing code quality is a constant battle for all developers so it makes sense to use every tool available.

Ian Cordasco’s flake8 is pretty much the standard in Python linting at the moment. It wraps three libraries: pyflakes (a static analyzer/linter), pep8 (a PEP8 checker) and McCabe (a cyclomatic complexity checker).

Running it against a project is dead simple. Just install it, go to your project root, and use the flake8 command:

If you want, you can run it yourself each time, but it is better to set up a pre-commit hook so that it runs every time you try to check in code and blocks the commit if it detects any quality problems.

There is a way to automatically install the hook (flake8 --install-hook), but I have found it unreliable, so I just manually add flake8 to the .git/hooks/pre-commit script. Here’s what my script looks like:

Remember that this script needs to be executable.

After you add that hook, your commits will be blocked if flake8 identifies any issues. The problem is that out of the box it is quite sensitive, so you might want to allow certain things to be committed that it thinks are problematic. There are two ways to do that.

For single lines that cause problems (because they are too long and don’t pass PEP8, for instance), you can add an ignore directive in a comment, like so:

Or if you want to ignore a certain type of error for the entire project, or exclude certain subdirectories, you can add a .flake8 config file and set up ignores and excludes in there. Here’s an example:

The ignore property takes a comma-separated list of error codes that should be ignored. In this case, E501 is the “line too long” error. The exclude property takes a comma-separated list of files and locations not to check. max-complexity specifies the maximum cyclomatic complexity allowed for functions, as determined by the McCabe library. The default value is 10, but I find that very conservative so I like to set it to 16.

That should be more than enough to get started, but if you want to learn more, you can check out the flake8 docs here.

What to do after Codecademy

Today I want to offer my opinion on a question that lots of people have asked me since I started this blog. “What should I do after I have finished an introductory course on Codecademy?”

Codecademy is basic stuff

You should be proud to have finished your first course and dipped your toes into programming. It’s a great achievement, but you also have to be realistic: this is just the beginning. You have made your way to base camp, but you haven’t yet attempted to reach the summit.

The fact is that Codecademy, while providing a solid grounding in the basics, doesn’t go much farther than that. On its own, it only teaches you a fraction of the things that you need to know to program professionally (if that’s your goal).

So be proud, but be humble. There is so much to learn that you will never know everything. And that’s a good thing, as long as you like learning.

Identify your goals

Maybe it’s obvious, but the path you take from now should be dictated by what you want to be able to do. If your goal is to write web applications, you should get to grips with frameworks such as Django, Flask and Bottle. If your goal is to use Python for data science, you should be learning things like Numpy, Scipy and Pandas (and hitting the math books hard). If you want to create desktop software, you should learn PyQt and Tkinter.

Once you have to make the decision about what area (or areas) to focus on, then go and find some beginner materials for them. I’m not going to list any here, because there are just so many, but rest assured that they are easy to find.

(That reminds me of another point: Google is your friend. Programmers like to joke that the job is 80% Googling, and there is some truth to that, depending on what you’re building. So you’d better improve your Google-fu and learn how to ask good questions.)

Projects are key

Working through courses and tutorials, and reading technical books, are important ways of improving your knowledge, but there is really no substitute for trying to build something and picking up what you need to know as you go along. The skills of identifying bugs quickly, deciphering arcane error messages and knowing when to stop fiddling with your code and move on are as important to programming as knowledge of advanced language features or algorithms.

With that in mind, I suggest you set yourself a project. Maybe you already have something in mind, in which case, awesome! Or maybe you don’t. In that case, take a look at this list of projects and pick one that seems fun and achievable.

http://www.dreamincode.net/forums/topic/78802-martyr2s-mega-project-ideas-list/

Learn some real computer science

Assuming you’re like a lot of the people picking up coding these days and your goal is to make web applications, then you can go a long way without even thinking about the theoretical underpinnings of programming.

Don’t be that guy. Apart from the often mooted practical considerations of writing efficient code or gaining access to higher paying jobs, there is a world of elegance and beauty in programming that you may not have expected. I am going to suggest a few resources for this because people have remarked that they don’t know where to start.

Code: The Hidden Language of Computer Hardware and Software

This is the amuse-bouche. Just read it one straight through for fun and try to become inspired!

NAND2Tetris

I can’t say enough good things about this course. It took me from a vague understanding of how computers executed my code to being able to conceptualize from the ground up how it all works. If you follow the course the whole way through you will build a (simulated) computer from first principles, create an assembler for it, a compiler for a Java-like language, and a basic operating system. Do it!

Coursera – Algorithms Part 1

This course, taught by Robert Sedgewick, will teach you a basic algorithmic toolbox. The algorithms is deals with are sort of like a “Greatest Hits” collection. Knowledge of these algorithms and design techniques behind forms a kind of lingua franca among programmers. For that reason, they are also really popular interview questions.

Phew! If you get to grips with all that stuff, you’ll be doing well!

Learn source control

I often see people advising beginners to learn Git before they do anything else. Git is a great program, and version control is one of the key skills needed to develop software at a professional level, but I can’t agree with the sentiment. The fact is that Git is extremely complicated. Even experienced programmers end up having to trawl the documentation and StackOverflow to figure out how to do certain things. So I suggest you learn Mercurial instead. It has many of the advantages of Git while being much more user-friendly. As a bonus, it’s written in Python!

They key thing you need to acquire at this stage of your education is a habit of using version control to track the development of your personal projects. You need to go through the process of changing code, breaking your whole project and being able to revert to a working state so that you can really understand why people use version control. For that purpose, the steep learning curve of Git is just going to put you off.

I’m not saying you shouldn’t learn Git later. You should, especially if you want to participate in open source projects. But at that point knowing Mercurial is only going to help, because you will already understand the concepts.

Learn an IDE and/or a text editor

Now is also the time to learn about IDEs and text editors. If you’ve been hanging around on programming forums (you should be), you will no doubt have got an inkling of the holy war between Vi and Emacs users, and the strong opinions of people who think IDEs are unnecessary bloatware. I don’t want to come down on either side of these arguments. I just want to suggest that you are now at the point where you should be investigating them for yourself.

But for my money, PyCharm is an amazing piece of software. 😀

The end

I hope I’ve given you something to think about with this article. If it all seems like a lot of work, well, it is! I’ll let Peter Norvig provide some perspective.

As always, I’m happy to answer your emails.