Table of Contents
- Professional Programming - about this list
- Principles
- Contributing to this list
- Must-read books
- Must-read articles
- Other general material and list of resources
- Topics
- Accounting
- Agentic coding
- AI (Artificial Intelligence)
- Algorithm and data structures
- API design & development
- Attitude, habits, mindset
- Authentication/authorization
- Automation
- Best practices
- Beyond software engineering & random
- Biases
- Business
- Buy vs. Build
- Cache
- Career growth
- Characters sets
- Chess
- Clouds
- Code reviews
- Coding & code quality
- Communication
- Compilers
- Configuration
- Continuous Integration (CI)
- Data analysis & data science
- Data semantics
- Databases
- Data formats
- Data science/data engineering
- Debugging
- Design (visual, UX, UI, typography)
- Design (OO modeling, architecture, patterns, anti-patterns, etc.)
- Dev environment & tools
- Docker
- Documentation
- Dotfiles
- Editors & IDE
- Engineering management
- Exercises
- Experimentation
- Fonts
- Functional programming (FP)
- Games development
- Generative AI
- Graphics
- Hardware
- HTTP
- Humor
- Incident response (oncall, alerting, outages, firefighting, postmortem)
- Internet
- Interviewing
- Kubernetes
- Large Language Model (LLM)
- Learning & memorizing
- Licenses (legal)
- Linux (system management)
- Low-code/no-code
- Low-level, assembly
- Machine learning/AI
- Math
- Marketing
- Network
- Observability (monitoring, logging, exception handling)
- Open source
- Operating system (OS)
- Over-engineering
- Performance
- Personal knowledge management (PKM)
- Personal productivity
- Perspective
- Privacy
- Problem solving
- Product management for software engineers
- Project management
- Programming languages
- Programming paradigm
- Public speaking (presenting)
- Reading
- Refactoring
- Regex
- Releasing & deploying
- Reliability
- Search
- Security
- Research papers
- Shell (command line)
- SQL
- State
- System administration
- System architecture
- Scalability
- Site Reliability Engineering (SRE)
- Technical debt
- Testing
- Tools
- Type system
- Typography
- Version control (Git)
- Work ethics, productivity & work/life balance
- Web development
- Writing (communication, blogging)
- Resources & inspiration for presentations
- Keeping up-to-date
- Concepts
- My other lists
Professional Programming - about this list
Give me six hours to chop down a tree and I will spend the first four sharpening the axe. (Abraham Lincoln)
A collection of full-stack resources for programmers.
The goal of this page is to make you a more proficient developer. You'll find only resources that I've found truly inspiring, or that have become timeless classics.
Principles
- This page is not meant to be comprehensive. I am trying to keep it light and not too overwhelming.
- The selection of articles is opinionated.
- I don't necessarily agree with or endorse every single line that is written in every single one of those resources. The same applies to their authors: I don't endorse everything each of those authors has said and will ever say.
Items:
- 🧰 : list of resources
- 📖 : book
- 🎞 : video/movie extract/movie/talk
- 🏙 : slides/presentation
- ⭐️ : must-read
- 📃 : paper
Contributing to this list
Feel free to open a PR to contribute!
I will not be adding everything: as stated above, I am trying to keep the list concise.
Must-read books
I've found these books incredibly inspiring:
- 📖 The Pragmatic Programmer: From Journeyman to Master: hands-on the most inspiring and useful book I've read about programming.
- 📖 Code Complete: A Practical Handbook of Software Construction: a nice addition to The Pragmatic Programmer, gives you the necessary framework to talk about code.
- 📖 Release It!: this books goes beyond code and gives you best practices for building production-ready software. It will give you about 3 years worth of real-world experience.
- 📖 Scalability Rules: 50 Principles for Scaling Web Sites
- 📖 The Linux Programming Interface: A Linux and UNIX System Programming Handbook: outside of teaching you almost everything you need to know about Linux, this book will give you insights into how software evolves, and the value of having simple & elegant interfaces.
- 📖 Structure and interpretation of Computer Programs (free): One of the most influential textbooks in Computer Science (written and used at MIT), SICP has been influential in CS education. Byte recommended SICP "for professional programmers who are really interested in their profession".
There are some free books available, including:
- 📖 Professional software development: pretty complete and a good companion to this page. The free chapters are mostly focused on software development processes: design, testing, code writing, etc. - and not so much about tech itself.
- 🧰 vhf/free-programming-books
- 🧰 EbookFoundation/free-programming-books
Must-read articles
- Practical Advice for New Software Engineers
- On Being A Senior Engineer
- Lessons Learned in Software Development: one of those articles that give you years of hard-earned lessons, all in one short article. Must read.
- Things I Learnt The Hard Way
- Spec first, then code
- Tests make better APIs
- Future thinking is future trashing
- Documentation is a love letter to your future self
- Sometimes, it's better to let the application crash than do nothing
- Understand and stay away of cargo cult
- "Right tool for the job" is just to push an agenda
- Learn the basics functional programming
- ALWAYS use timezones with your dates
- ALWAYS use UTF-8
- Create libraries
- Learn to monitor
- Explicit is better than implicit
- Companies look for specialists but keep generalists longer
- The best secure way to deal with user data is not to capture it
- When it's time to stop, it's time to stop
- You're responsible for the use of your code
- Don't tell "It's done" when it's not
- Pay attention on how people react to you
- Beware of micro-aggressions
- Keep a list of "Things I Don't Know"
- Signs that you're a good programmer (not everything in here is great - some of the points are counterproductive)
- The instinct to experiment first
- Emotional detachment from code and design
- Eager to fix what isn't broken
- Fascinated by the incomprehensible
- Compelled to teach
- Incorruptible patience
- A destructive pursuit of perfection
- Encyclopedic grasp of the platform
- Thinks In Code
- When In Rome, Does As Romans Do
- Creates their own tools
- Indifferent to Hierarchy
- Excited by failure
- Indifferent to circumstances
- Substitutes impulse for commitment
- Driven by experiences
- 7 absolute truths I unlearned as junior developer
- Early in your career, you can learn 10x more in a supportive team in 1 year, than coding on your own
- Every company has problems, every company has technical debt.
- Being overly opinionated on topics you lack real-world experience with is pretty arrogant.
- Many conference talks cover proof of concepts rather than real-world scenarios.
- Dealing with legacy is completely normal.
- Architecture is more important than nitpicking.
- Focus on automation over documentation where appropriate.
- Having some technical debt is healthy.
- Senior engineers must develop many skills besides programming.
- We’re all still junior in some areas.
- How to Build Good Software
- A good high-level summary of fundamental engineering practices.
- The root cause of bad software has less to do with specific engineering choices, and more to do with how development projects are managed.
- There is no such thing as platonically good engineering: it depends on your needs and the practical problems you encounter.
- Software should be treated not as a static product, but as a living manifestation of the development team’s collective understanding.
- Software projects rarely fail because they are too small; they fail because they get too big.
- Beware of bureaucratic goals masquerading as problem statements. If our end goal is to make citizens’ lives better, we need to explicitly acknowledge the things that are making their lives worse.
- Building software is not about avoiding failure; it is about strategically failing as fast as possible to get the information you need to build something good.
- How to be a -10x Engineer
- Nullify the output of 10 engineers.
- Hold 10 engineers hostage in a technical discussion.
- Waste 10 weeks of wages on cloud costs.
- Waste 400 hours of engineering on bad architecture.
- Incur 400 hours of bug triage.
- A Bunch of Programming Advice I'd Give To Myself 15 Years Ago
- If you (or your team) are shooting yourselves in the foot constantly, fix the gun
- Assess the trade-off you’re making between quality and pace, make sure it’s appropriate for your context
- Spending time sharpening the axe is almost always worth it
- If you can’t easily explain why something is difficult, then it’s incidental complexity, which is probably worth addressing
- Try to solve bugs one layer deeper
- Don’t underestimate the value of digging into history to investigate some bugs
- Bad code gives you feedback, perfect code doesn’t. Err on the side of writing bad code
- Make debugging easier
- When working on a team, you should usually ask the question
- Shipping cadence matters a lot. Think hard about what will get you shipping quickly and often
- Expert Generalists, martinfowler.com, proposes an interesting take on the "T-shaped engineer"
- The Characteristics of an Expert Generalist: Curiosity, Collaborativeness, Customer Focus, Favor Fundamental Knowledge, Blend of Generalist and Specialist Skills, Sympathy for Related Domains
- Assessing Expert Generalists: hiring and career progression
- Growing Expert Generalists: From Tools to Fundamentals
- "Why does our attention keep drifting toward tool expertise? It isn't because people are shortsighted or lazy; it's because the fundamentals are hard to see amid the noise."
- Expert Generalists still need Specialists
- Expert Generalists in the Age of LLMs
- "Similarly to a specialist, an LLM can rapidly answer questions that an Expert Generalist will have when working in a new domain."
- "Rather than looking for “the answer”, they prompt them to generate questions, explaining mechanisms, and providing examples and even tools that help explore the underlying mechanisms of an idea."
Other general material and list of resources
Other lists
Books
- 📖 The Imposter's Handbook - $30. From the author: "Don't have a CS Degree? Neither do I - That's why I wrote this book."
- 📖 The Computer Science Book
- 📖 The Software Engineer's Guidebook: Gergely Orosz's guidebook to the software engineering industry. Extremely complete.
Articles
- mr-mig/every-programmer-should-know: a collection of (mostly) technical things every software developer should know
- Famous Laws Of Software Development
- The Amazon Builders' Library
- There is a list of the best articles in this Twitter Thread
- kdeldycke/awesome-falsehood: Falsehoods Programmers Believe in
- hellerve/programming-talks
- TechYaks: list of talks
- Talks that changed the way I think about programming
- What every computer science major should know
- kamranahmedse/developer-roadmap
- mtdvio/every-programmer-should-know: a collection of (mostly) technical things every software developer should know about
- Mike Acton’s Expectations of Professional Software Engineers
- Things they didn't teach you about Software Engineering
- Domain knowledge is more important than your coding skills
- Code is secondary. Business value is first.
- You work with uncertainty most of the time
- We overestimate our short-term ability, but underestimate our long-term ability.
- Specialisation is for insects.
- Want an unfair advantage in your tech career? Consume content meant for other roles
- Cross-functional understanding is critical in modern tech companies
- Helps to avoid underestimating the importance and difficulty other roles
- Helps you to be strategic in your interaction with people in that role
- Teach Yourself Programming in Ten Years
- Mistakes You Apparently Just Have to Make Yourself
- The Best Programmers I Know
- Laws of Software Engineering
Axioms
- Precepts - Urbit
- Data is better than code.
- Correctness is more important than performance.
- Deterministic beats heuristic.
- One hundred lines of simplicity is better than twenty lines of complexity.
- If your abstractions are leaking, it's not due to some law of the universe; you just suck at abstracting. Usually, you didn't specify the abstraction narrowly enough.
- If you avoid changing a section of code for fear of awakening the demons therein, you are living in fear. If you stay in the comfortable confines of the small section of the code you wrote or know well, you will never write legendary code. All code was written by humans and can be mastered by humans.
- If there's clearly a right way to do something and a wrong way, do it the right way. Coding requires incredible discipline.
- The best way to get the right answer is to try it the wrong way.
- Practice tells you that things are good or bad; theory tells you why.
- Not being qualified to solve a problem is no reason not to solve it.
- If you don't understand a system you're using, you don't control it. If nobody understands the system, the system is in control.
- Embedded Rules of Thumb
- 50 Ideas That Changed My Life
- Reflections on 10,000 Hours of Programming
- 20 Things I've Learned in my 20 Years as a Software Engineer
Courses
- Google Tech Dev Guide
- The Missing Semester of Your CS Education, MIT. Includes lectures about the shell, editors, data wrangling, git, debugging and profiling, meta programming, security and cryptography.
- Mathematics for the adventurous self-learner, Neil Sainsbury
- jwasham/coding-interview-university: a complete computer science study plan to become a software engineer.
- Teach Yourself Computer Science: an opinionated set of the best CS resources.
- ossu/computer-science: free self-taught education in Computer Science!
Topics
Accounting
Agentic coding
AI (Artificial Intelligence) & ML (Machine Learning)
Agentic coding & best pratices
How AI works
- Transformers from Scratch
- A Gentle Introduction to Graph Neural Networks
- The Math Behind Artificial Intelligence, Tiago Monteiro. Covers linear algebra, calculus, probability, and optimization theory in the context of AI and machine learning.
- The State of Context Engineering summarizes the relevant approaches to context management: context augmentation, context quarantine, context summarization ("compaction"), tool loadout, context pruning, context offloading.
Limits of genAI
- Why I Think You Should Almost Never Use AI to Write Anything Substantive
- When AI builds itself, Anthropic
Algorithm and data structures
- Read the CLRS. You can watch and download the course on OCW - there are newer courses as well.
- Or The Algorithm Design Manual (Skiena)
- Try out some algorithms on Project Euler
- CS 61B Spring 2023
Other resources:
- Algorithms, Jeff Erickson
Here are some useful & interesting algo & DS visualizations:
- Grokking Algorithms
- Essential Algorithms
- Data Structure Visualization
- 🎞 15 Sorting Algorithms in 6 Minutes
- Hashing
- Visualizing Algorithms
- B-trees and database indexes
- Big O visualizations
- Algorithm explained like Ikea instructions
- Decision Trees
Example implementations:
- trekhleb/javascript-algorithms: algorithms and data structures implemented in JavaScript
- The Algorithms
Algorithms in distributed systems:
API design & development
General REST content:
- Architectural Styles and the Design of Network-based Software Architectures, Roy Fielding (the inventor of REST)
- A collection of useful resources for building RESTful HTTP+JSON APIs.
- Best practices for REST API design, Stack Overflow Blog
- 📖 Undisturbed REST: a guide to designing the perfect API: very complete book about RESTful API design.
Example guidelines:
- Microsoft's Rest API guidelines
- Zalando RESTful API and Event Scheme Guidelines
- Google's API Design Guide: a general guide to design networked API.
- AIP-1: AIP Purpose and Guidelines
- AIP stands for API Improvement Proposal, which is a design document providing high-level, concise documentation for API development.
More specific topics:
- Why you should use links, not keys, to represent relationships in APIs, Martin Nally, Google
- "Using links instead of foreign keys to express relationships in APIs reduces the amount of information a client needs to know to use an API, and reduces the ways in which clients and servers are coupled to each other."
- Give me /events, not webhooks
- Events can unlock much-needed webhook features, like allowing your webhook consumers to replay or reset the position of their webhook subscription.
- Unlocking the Power of JSON Patch
Attitude, habits, mindset
- Mastering Programming, Kent Beck.
- The traits of a proficient programmer
- The tao of programming: a set of parables about programming.
- Taking Ownership Is The Most Effective Way to Get What You Want
- Finding Time to Become a Better Developer
- Ten minutes a day: how Alex Allain wrote a book in less than 200 hours, by writing 10 minutes every day.
- The care and feeding of software engineers (or, why engineers are grumpy)
- In the triumvirate of software, product managers, designers, and software engineers, only the engineers are expected to turn off their creative minds and just produce.
- Both engineers and product managers tend to think, incorrectly, that product specifications or requirements are equivalent to the furniture manual from Ikea.
- This is one of the top things that make engineers grumpy: constantly shifting priorities.
- Even though many engineers will complain that product managers change their minds, almost none will account for that in their time estimates.
- Computer science programs aren’t about preparing you for the tasks you’ll face in industry.
- When there are more engineers than can be used, engineering time ends up going away from developing and towards planning, synchronization, and coordination.
- Involve engineers in the creative process
- Give engineers opportunities to be creative.
- Encourage time off.
- Let 'em code
- Express appreciation
- The Product-Minded Software Engineer, Gergely Orosz
- Great product engineers know that minimum lovable products need the right depth
- Product-minded engineers quickly map out edge cases and think of ways to reduce work on them: often bringing solutions that require no engineering work
- Engage in user research and customer support
- Bring well-backed product suggestions to the table
- Offer product/engineering tradeoffs
- 40 Lessons From 40 Years, Steve Schlafman
- If you want to make progress on the things that matter most, you need to decide who you’re going to disappoint. It’s inevitable.
- The best investment you can make is your own education. Never stop learning. The second best investment you can make is building your network through authentic and meaningful interactions. It is what you know and who you know.
- You’ll never get what you don’t ask for or actively seek out. Go for it!
- It’s not about the light at the end of the tunnel. It’s the tunnel. Show up every day and enjoy the process.
- A great teammate always puts the organization and its purpose ahead of their own self interests.
- Pick your spots. We have limited time and our brains can only process so much. Focus is key. Choose wisely.
- Every person is likely struggling with something. Be kind. Be helpful.
- On Coding, Ego and Attention
- Beginner’s mind accepts the fact that absolute knowledge is infinite and thus keeping score is a waste of time.
- Mastery is simply the accumulation of momentum, not the accumulation of knowledge.
- Dealing with ego distraction has taught me to love the problem solving process. It’s taught me to love and respect the learning process. As a result I’m more productive. I’m less anxious. I’m a better teammate. I’m a better friend and a better thinker.
- Fixed vs. Growth: The Two Basic Mindsets That Shape Our Lives
- What does a great software engineer look like?
- Good sleep, good learning, good life
- 🎞 Steve Jobs: if you don't ask for help, you won't get very far
- Programming quotes
- Be Kind
- Being kind is fundamentally about taking responsibility for your impact on the people around you.
- It requires you be mindful of their feelings and considerate of the way your presence affects them.
- Warren Buffett Says This 1 Simple Habit Separates Successful People From Everyone Else
- The difference between successful people and really successful people is that really successful people say no to almost everything.
- How to get lucky?
- Programmers should stop celebrating incompetence, DHH
- The magic of programming is largely just things you don't know yet.
- It's not fine to think you shouldn't be on some paths towards mastery, if you intend to make programming your career.
- There’s no speed limit
- Don't Wait for Motivation, Act for Momentum
- Start with a tiny task. Then ride its momentum.
- The Most Important Coding Habits
- The Most Important Coding Habits
- Daily stretches
- Take regular breaks
- Don’t code late at night
- Improve your coding environment
- Advice for new software devs who've read all those other advice essays
- Microservices aren't the problem. Incompetent people are
- High Agency (30-min read)
- What is "good taste" in software engineering?: it's all about contextual tradeoff.
Imposter syndrome is underrated: a lot of talk goes into overcoming imposter syndrome. I say embrace self-skepticism and doubt yourself every day. In a fast-moving industry where lots of your knowledge expires every year, even the most junior people around you constantly cook up skills you don't have; you stay competitive by applying with the determination (and even fear) of the novice. The upside of this treadmill is that every engineer is on it: just because you're an imposter doesn't mean that other people are more deserving than you, because they're imposters too. You should advocate for yourself, take risks, pat yourself on the back when things go well, and, as you start to build a track record of solving problems, trust your skills and adaptability. Just make no mistake: you're only as good as the last problem you solve.
Dan Heller, Building a Career in Software
I had learned already never to empty the well of my writing, but always to stop when there was still something there in the deep part of the well, and let it refill at night from the springs that fed it. -- Ernest Hemingway
- The Grug Brained Developer: habits of self-aware programmer. Like Tao of Programming, different style.
Good judgment comes from experience. Experience comes from bad judgment.
Procrastination
- News is bad for you – and giving up reading it will make you happier, The Guardian
- News misleads
- News is irrelevant
- News has no explanatory power
- News is toxic to your body
- News increases cognitive errors
- News inhibits thinking
- News works like a drug
- News wastes time
- News makes us passive
- News kills creativity
Authentication/authorization
- Authorization in a microservices world
- Authorization Logic: Rules are hard because they evolve over time
- The Copenhagen Book provides a general guideline on implementing auth in web applications
Automation
Best practices
Beyond software engineering & random
Biases
Biases don't only apply to hiring. For instance, the fundamental attribution bias also applies when criticizing somebody's code written a long time ago, in a totally different context.
- Cognitive bias cheat sheet. #hiring
Business
- Payments 101 for a Developer
- The 4 biggest problems with homemade billing systems
- 🦑 The 14 pains of building your own billing system
Buy vs. Build
- Choose Boring Technology
- Build vs. Buy
- The reason we want to buy as much as possible is that an organisation has a limited capacity for expertise, so we don’t want to have to become experts on things that don’t make up a competitive advantage.
- Platform Engineering: Build vs Buy
- If someone tells me they can build something cheaper than a vendor, I’m immediately skeptical because I don’t think most people can accurately forecast the actual cost of maintenance in the long term.
Cache
- Caching challenges and strategies, Amazon Builders Library
Career growth
- The Conjoined Triangles of Senior-Level Development looks into how to define a senior engineer.
- Ten Principles for Growth as an Engineer, Dan Heller.
- Don't Call Yourself a Programmer, Patrick McKenzie.
- On being an Engineering Manager
- The career advice I wish I had at 25
- A career is a marathon, not a sprint
- Most success comes from repetition, not new things
- If work was really so great all the rich people would have the jobs
- Management is about people, not things
- Genuinely listen to others
- Recognise that staff are people with finite emotional capacity
- Don’t just network with people your own age
- Never sacrifice personal ethics for a work reason
- Recognise that failure is learning
- Career advice I wish I’d been given when I was young
- Don’t focus too much on long-term plans.
- Find good thinkers and cold-call the ones you most admire.
- Assign a high value to productivity over your whole lifespan.
- Don’t over-optimise things that aren’t your top priority.
- Read a lot, and read things that people around you aren’t reading.
- Reflect seriously on what problem to prioritise solving.
- Read more history.
- Why Good Developers are Promoted into Unhappiness, Rob Walling. Or why management might not be for you.
- A guide to using your career to help solve the world’s most pressing problems
- What's a senior engineer's job? You need to be more than just an individual contributor.
- From Coding Bootcamp Graduate to Building Distributed Databases
- Read Books (and papers), not Blog Posts
- Take responsibility for your career trajectory
- 🏙 The Well Rounded Engineer includes lots of great book recommendations.
- Paradigm polyglot (learn different languages & paradigms)
- Database polyglot
- Protocol polyglot (preferably TCP/IP and HTTP)
- Proficiency with build tooling, packaging and distribution
- Debugging, observability
- Deployment, infra and devops
- Software architecture and scaling
- Ability to write toy compilers, interpreters and parsers
- Ability to write toy games
- Ability to understand algorithmic analysis
- Some career advice, Will Larson.
- Advice you get is someone’s attempt to synthesize their experiences, not an accurate statement about how the world works.
- Build a reservoir of prestige.
- Some folks are so good at something that they end up being irreplaceable in their current role, which causes them to get stuck in their role even if they’re a good candidate for more interesting ones.
- Great relationships will follow you everywhere you go. Bad ones too.
- Early in your career, try to work at as many different kinds of companies and in different product vertical as you can.
- Evil tip: avoid "easy" things
- The Ultimate Code Kata
- Traits of a senior software engineer: impact, perception, visibility, influence, mentoring
- Software Engineering - The Soft Parts
- Think critically and formulate well-reasoned arguments
- Master the fundamentals
- Focus on the user and all else will follow
- Learn how to learn
- How To Own Your Growth As A Software Engineer
- The Forty-Year Programmer
- The Better You Get, the Less You Look Like Everybody Else
- You Learn Deep Principles by Doing the Basics
- Look to Other Fields, Learn From Other Fields
- Be Careful About Productivity Tips
- Senior Engineers are Living in the Future
- What would a map of your career look like?
- How to be successful at Amazon (or any other large company for that matter)
- Being good isn’t enough
- The biggest gains come from combining disciplines.
- Stop Avoiding Politics is a good contrarian take on the current (negative) connotation of the word "politics".
- In a way, it is unfortunate that "politics" got that connotation since Aristotle rightfully considered politics the highest form of practical wisdom, since it has to do with humans as social animals.
- "Good politics is just being strategic about relationships and influence in the service of good outcomes."
- Examples: building relationships before you need them, understanding the incentives, managing up effectively, creating win-win situation, being visible.
- "The alternative to good politics isn’t no politics. It’s bad politics winning by default. It’s the loud person who’s wrong getting their way because the quiet person who’s right won’t speak up."
- 21 Lessons from 14 Years at Google
- Publishing your work increases your luck
About senior engineers:
Choosing your next/first opportunity
Getting to Staff Eng
- I became a FAANG Staff Engineer in 5 years. These are the 14 lessons I learned along the way.
- Software engineering isn’t just coding. Actually, coding is a small part of it.
- Pipeline your work
- Be open to feedback and listen. Like, seriously, listen.
- Great feedback is hard to find; treasure it.
- Keep an eye on the horizon (but not both).
- Figure out what matters and let the rest go.
- Comparison really is the thief of joy.
- Mentorship is a beautiful thing.
- Good days, in general, don’t just “happen”.
- Advice and guidance are just that; they aren’t rules.
- Guides for reaching Staff-plus engineering roles, Will Larson
- Staff archetypes, Will Larson
- How I Find Problems to Solve as a Staff Engineer
Characters sets
- The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
- The Absolute Minimum Every Software Developer Must Know About Unicode in 2023 (Still No Excuses!)
Chess
(yes - chess gets its own section :)
Clouds
- open-guides/og-aws: a practical guide to AWS
Code reviews
- How to do a code review, Google's engineering practices documentation.
- Post-Commit Reviews: an interesting idea to increase developer velocity (there are some caveats though).
- How to Make Your Code Reviewer Fall in Love with You
- Review your own code first
- Write a clear changelist description
- Automate the easy stuff
- Answer questions with the code itself
- Narrowly scope changes
- Separate functional and non-functional changes
- Respond graciously to critiques
- Artfully solicit missing information
- Award all ties to your reviewer
- Minimize lag between rounds of review
- How to Do Code Reviews Like a Human
- Ask HN: How do you review code?: great discussion on HackerNews, full of interesting ideas.
- Maslow's pyramid of code reviews
- Another one on the same topic: The Code Review Pyramid
- Code review in remote teams: very complete set of rules.
- No code reviews by default
- Responsibility over convention
Coding & code quality
- Write code that is easy to delete, not easy to extend
- The Ten Commandments of Egoless Programming
- 📖 Clean Code: A Handbook of Agile Software Craftsmanship, Robert C. Martin. Describes numerous useful best practices. A bit long. There's also a clean code cheatsheet.
- What Software Craftsmanship is about
- We’re tired of writing crap.
- We will not accept the stupid old lie about cleaning things up later.
- We will not believe the claim that quick means dirty.
- We will not allow anyone to force us to behave unprofessionally.
- Tips on naming boolean variables
- There is a convention to prefix boolean variables and function names with "is" or "has".
- Try to always use is, even for plurals (
isEachUserLoggedInis better thanareUsersLoggedInorisUsersLoggedIn) - Avoid custom prefixes (
isPaidForis better thanwasPaidFor) - Avoid negatives (
isEnabledis better thanisDisabled)
- How To Write Unmaintainable Code
- kettanaito/naming-cheatsheet: : comprehensive language-agnostic guidelines on variables naming. Home of the A/HC/LC pattern.
- 🧰 Quality Engineering Guides
- What Makes Code Hard To Read: Visual Patterns of Complexity
- zakirullin/cognitive-load: 🧠 Cognitive Load is what matters
- How to create software quality
Communication
See also the Writing section
- How to communicate effectively as a developer
- Lots of concrete advice and examples for short, medium and long-form writing
- What Do You Visualize While Programming?
Compilers
- The Compiler Writer Resource Page
- kanaka/mal: mal - Make a Lisp
- Let's Build a Compiler, Jack W. Crenshaw, 1988
- Writing a C Compiler, in Zig
Configuration
- The downsides of JSON for config files, Martin Tournoij.
- Can't add comments
- Excessive quotation and syntax noise
- Using DC (declarative configuration) to control logic is often not a good idea.
- Your configs suck? Try a real programming language
- Most modern config formats suck
- Use a real programming language
- Code rant: The Configuration Complexity Clock
- I’m not saying that it’s never appropriate to implement complex configuration, a rules-engine or a DSL, Indeed I would jump at the chance of building a DSL given the right requirements, but I am saying that you should understand the implications and recognise where you are on the clock before you go down that route.
- Initially there was hope that non-technical business users would be able to use the GUI to configure the application, but that turned out to be a false hope; the mapping of business rules into the engine requires a level of expertise that only some members of the development team possess.
Continuous Integration (CI)
- Continuous Integration, MartinFowler.com
Data analysis & data science
- Ways to make fake data look meaningful
- Don’t share the raw data
- Don’t share your methodology
- Don’t include confidence intervals
- Don’t challenge your own data
- 📖 Python Data Science Handbook, O'Reilly
- The Shape and Feel of the Post-AI Data Stack
Data semantics
- Making Your Data Ready for Agentic AI, martinfowler.com
Databases
See also the SQL section.
- A plain English introduction to CAP Theorem
- PACELC theorem: "in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C) (as per the CAP theorem), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C)."
- Zero downtime database migrations (code examples are using Rails but this works great for any programming language)
- Readings in Database Systems, 5th Edition
- Comparing database types: how database types evolved to meet different needs
- How does a relational database work
- Use the index, Luke
- Course introduction — MySQL for Developers, PlanetScale
- Why you should probably be using SQLite | Epic Web Dev
- 📖 Database Internals
Scaling databases:
- How Figma's Databases Team Lived to Tell the Scale: interesting story about sharding
Internals
- How Query Engines Work
- Let's Build a Simple Database
- Algorithms Behind Modern Storage Systems, ACM Queue
- sorted string tables (SST) from first principles
NoSQL
- NOSQL Patterns
- NoSQL Databases: a Survey and Decision Guidance
- The DynamoDB docs has some great pages:
- Redis Explained
Postgres
- Safe Operations For High Volume PostgreSQL (this is for PostgreSQL but works great for other DBs as well).
- Transaction Isolation in Postgres, explained
- PostgreSQL exercises
- Postgres operations cheat sheet
- Postgres: don't Do This
- PostgreSQL and UUID as primary key
- Postgres lock explained
"Just use postgres":
Data formats
- Falsehoods Programmers Believe About Phone Numbers, Google's
libphonenumber. - Rules for Autocomplete: rough specifications for autocomplete fields
- Falsehoods programmers believe about addresses
- Falsehoods Programmers Believe About Names
- kdeldycke/awesome-falsehood: falsehoods programmers believe in
- Understanding UUIDs, ULIDs and String Representations
- Falsehoods Programmers Believe About Falsehoods Lists
- Australia/Lord_Howe is the weirdest timezone
- A love letter to the CSV format
- Falsehoods Programmers Believe About Aviation
- Schemas - Schema.org
- ZIP Code First
Data science/data engineering
- A dirty dozen: twelve common metric interpretation pitfalls in online controlled experiments
- datastacktv/data-engineer-roadmap: roadmap to becoming a data engineer
- Awesome Data Engineering Learning Path
- Emerging Architectures for Modern Data Infrastructure
- How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh
- Data platforms based on the data lake architecture have common failure modes that lead to unfulfilled promises at scale.
- We need to consider domains as the first class concern, apply platform thinking to create self-serve data infrastructure, and treat data as a product.
- MLOps
- Uber's Big Data Platform: 100+ Petabytes with Minute Latency
- SQL should be the default choice for data transformation logic
Debugging
Also see the Incident Response section in this doc
- Rubber Duck Problem Solving
- Rubber Ducking
- Five Whys
- The Five Lies Analysis
- The real problem reveals itself when the technique becomes a part of a template.
- Action items can be very distant from the root cause.
- Related article: The Evolution of SRE at Google
- The Infinite Hows criticizes the five whys method and advocates for a different set of questions to learn from the most from incidents.
- See also: Human errors: models and management
- "The issue with the Five Whys is that it’s tunnel-visioned into a linear and simplistic explanation of how work gets done and events transpire."
- "Human error becomes a starting point, not a conclusion." (Dekker, 2009)
- "When we ask 'how?', we’re asking for a narrative."
- "When it comes to decisions and actions, we want to know how it made sense for someone to do what they did."
- At each "why" step, only one answer will be selected for further investigation. Asking "how" encourage broader exploration.
- "In accident investigation, as in most other human endeavours, we fall prey to the What-You-Look-For-Is-What-You-Find or WYLFIWYF principle. This is a simple recognition of the fact that assumptions about what we are going to see (What-You-Look-For), to a large extent will determine what we actually find (What-You-Find)." (Hollnagel, 2009, p. 85) (see illustration of WYLFIWYF)
- "A final reason why a 'root cause' may be selected is that it is politically acceptable as the identified cause. Other events or explanations may be excluded or not examined in depth because they raise issues that are embarrassing to the organization or its contractors or are politically unacceptable." (Nancy Leveson, Engineering a Safer World, p. 20)
- Bounded rationality: rational individuals will select a decision that is satisfactory rather than optimal
- The article provide concrete ways and questions to solicit stories from people, which will yield better insights.
- What were you expecting to happen?
- If you had to describe the situation to your colleague at that point, what would you have told?
- Did this situation fit a standard scenario?
- What were you trying to achieve?Were there multiple goals at the same time?Was there time pressure or other limitations on what you could do?
- See template here
- Linux Performance Analysis in 60,000 Milliseconds
- Post-Mortems at HubSpot: What I Learned From 250 Whys
- Debugging zine, Julian Evans
- If you understand a bug, you can fix it
- The Thirty Minute Rule: if anyone gets stuck on something for more than 30 minutes, they should ask for help
- How to create a Minimal, Reproducible Example, Stack Overflow
- Some ways to get better at debugging, Julia Evans
- Learn the codebase
- Learn the system (e.g., HTTP stack, database transactions)
- Learn your tools (e.g.,
strace,tcpdump) - Learn strategies (e.g., writing code to reproduce, adding logging, taking a break)
- Get experience: according to a study, "experts simply formed more correct hypotheses and were more efficient at finding the fault."
- What exactly is the 'Saff Squeeze' method of finding a bug?
- A systematic technique for deleting both test code and non-test code from a failing test until the test and code are small enough to understand.
- tcpdump is amazing, Julia Evans
- What we talk about when we talk about ‘root cause’
- David A. Wheeler's Review of "Debugging" by David J. Agans
- Troubleshooting: The Skill That Never Goes Obsolete
- Includes links to interesting debugging stories
- Falsehoods software teams believe about user feedback
Design (visual, UX, UI, typography)
I highly recommend reading The Non-Designer's Design Book. This is a pretty short book that will give you some very actionable design advice.
- If you're working on data, Edward Tufte's The Visual Display of Quantitative Information is considered a classic.
- The Universal Principles of Design will give you enough vocabulary and concepts to describe design challenges into words.
- Book recommendations from HackerNews
- 🏙Design for Non-Designers
Articles :
- 10 Usability Heuristics Every Designer Should Know
- Visibility of System Status
- The Match Between The System And The Real World
- Every system should have a clear emergency exit
- Don't forget that people spend 90% of their time interacting with other apps
- Recognition Rather Than Recall (recognition = shallow form of retrieval from memory, e.g. a familiar person, recall = deeper retrieval)
- ”Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” – Antoine de Saint-Exupery
- Help Users Recognize, Diagnose, And Recover From Errors
- How to pick more beautiful colors for your data visualizations
- Visual design rules you can safely follow every time
- Malleable software: Restoring user agency in a world of locked-down apps
Typograhy: see "Typography" section
Resources:
- 🧰 bradtraversy/design-resources-for-developers: design and UI resources from stock photos, web templates, CSS frameworks, UI libraries, tools...
- Interfaces › Cheat Sheet
Design (OO modeling, architecture, patterns, anti-patterns, etc.)
Here's a list of good books:
- 📖 Design Patterns: Elements of Reusable Object-Oriented Software: dubbed "the gang of four", this is almost a required reading for any developer. A lot of those are a bit overkill for Python (because everything is an object, and dynamic typing), but the main idea (composition is better than inheritance) definitely is a good philosophy.
- And their nefarious nemesis Resign Patterns
- 📖 Patterns of Enterprise Application Architecture: learn about how database are used in real world applications. Mike Bayer's SQLAlchemy has been heavily influenced by this book.
- 📖 Domain-Driven Design: Tackling Complexity in the Heart of Software, Eric Evans
- 📖 Clean Architecture, Robert C. Martin. Uncle Bob proposes an architecture that leverages the Single Responsibility Principle to its fullest. A great way to start a new codebase. Also checkout the clean architecture cheatsheet and this article.
- 📖 Game Programming Patterns: a book about design, sequencing, behavioral patterns and much more by Robert Nystrom explained through the medium of game programming. The book is also free to read online here.
One of the absolute references on architecture is Martin Fowler: checkout his Software Architecture Guide.
Articles:
- O'Reilly's How to make mistakes in Python
- Education of a Programmer: a developer's thoughts after 35 years in the industry. There's a particularly good section about design & complexity (see "the end to end argument", "layering and componentization").
- Domain-driven design, Wikipedia.
- On the Spectrum of Abstraction 🎞, Cheng Lou
- The “Bug-O” Notation, Dan Abramov
- Antipatterns
- Inheritance vs. composition: a concrete example in Python. Another slightly longer one here. One last one, in Python 3.
- Composition Instead Of Inheritance
- Complexity and Strategy: interesting perspective on complexity and flexibility with really good examples (e.g. Google Apps Suite vs. Microsoft Office).
- The Architecture of Open Source Applications
- The Robustness Principle Reconsidered
- Jon Postel: "Be conservative in what you do, be liberal in what you accept from others." (RFC 793)
- Two general problem areas are impacted by the Robustness Principle: orderly interoperability and security.
- Basics of the Unix Philosophy, Eric S Raymond
- Eight Habits of Expert Software Designers: An Illustrated Guide
- No Silver Bullet - Essence and Accident in Software Engineering, Frederick P. Brooks, Jr. (1986)
- There are four properties of software systems which make building software hard: Complexity, Conformity, Changeability and Invisibility
- There are ways to address this:
- Exploiting the mass market to avoid constructing what can be bought. ("Buy vs. Build")
- Using rapid prototyping as part of a planned iteration in establishing software requirements.
- Growing software organically, adding more and more function to systems as they are run, used, and tested
- Identifying and developing the great conceptual designers of the rising generation.
- (also included in The Mythical Man-Month)
- Out of the Tar Pit, Ben Moseley, Peter Marks (2006) introduces the distinction between essential and accidental complexity
- Complexity is the root cause of the vast majority of problems with software today. Unreliability, late delivery, lack of security — often even poor performance in large-scale systems can all be seen as deriving ultimately from unmanageable complexity.
- Quoting Djikstra: "testing is hopelessly inadequate....(it) can be used very effectively to show the presence of bugs but never to show their absence."
- Functional programming goes a long way towards avoiding the problems of state-derived complexity, thanks to immutability and clear separation of state and logic.
- A Note on Essential Complexity
- The goal of the software engineer is to minimize accidental complexity and assist with essential complexity.
- Software Design is Knowledge Building
- Programming should be regarded as an activity by which the programmers form or achieve a certain kind of insight, a theory, of the matters at hand. This suggestion is in contrast to what appears to be a more common notion, that programming should be regarded as a production of a program and certain other texts.
- The building of the program is the same as the building of the theory of it by the team of programmers.
- Cognitive load is what matters
- A well-crafted monolith with truly isolated modules is often much more flexible than a bunch of microservices.
- Three decades on, microkernel-based GNU Hurd is still in development, and monolithic Linux is everywhere
- "Reduce cognitive load by limiting the number of choices." (Rob Pike)
- The same rule applies to all sorts of numeric statuses (in the database or wherever) - prefer self-describing strings.
- With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. (Hyrum's Law)
- DDD is about problem space, not about solution space.
- Familiarity is not the same as simplicity
- The more mental models there are to learn, the longer it takes for a new developer to deliver value.
- CUPID: for joyful coding
- Composable: plays well with others
- Unix philosophy: does one thing well
- Predictable: does what you expect
- Idiomatic: feels natural
- Domain-based: the solution domain models the problem domain in language and structure
- Nobody Gets Promoted for Simplicity
“Simplicity is a great virtue, but it requires hard work to achieve and education to appreciate. And to make matters worse, complexity sells better.” — Edsger Dijkstra
You can use an eraser on the drafting table or a sledge hammer on the construction site. (Frank Lloyd Wright)
Resources:
Design: database schema
- A humble guide to database schema design, Mike Alche
- Use at least third normal form
- Create a last line of defense with constraints
- Never store full addresses in a single field
- Never store firstname and lastname in the same field
- Establish conventions for table and field names.
- YAGRI: You are gonna read it: store
created_at,created_byetc.
Design: patterns
- KeystoneInterface, Martin Fowler.
- Build all the back-end code, integrate, but don't build the user-interface
- 101 Design Patterns & Tips for Developers
- Python Design Patterns: For Sleek And Fashionable Code: a pretty simple introduction to common design patterns (Facade, Adapter, Decorator). A more complete list of design patterns implementation in Python on Github.
- SourceMaking's Design Patterns seems to be a good web resource too.
- Anti-If: The missing patterns
Design: simplicity
- Simple Made Easy 🎞, Rich Hickey. This is an incredibly inspiring talk redefining simplicity, ease and complexity, and showing that solutions that look easy may actually harm your design.
Dev environment & tools
Tools
- Glances: An eye on your system
- HTTPie: a CLI, cURL-like tool for humans
- jq: command-line JSON processor
- tmux: terminal multiplexer
- htop: an interactive process viewer for Linux
- htop explained (all in one page)
- socat
- Visual guide to SSH tunnels
- casey/just: a command runner written in Rust (claims to be better than Makefile)
- Gazr: an opinionated way to define your
Makefile
Article about tools:
- The return of fancy tools
- Simple tools make you think a little more
- Drucker: "I’m not writing it down to remember it later, I’m writing it down to remember it now."
- Frictionless note-taking produces notes, but it doesn't produce memory.
Docker
See also the Python-specific section in charlax/python-education.
- Best Practices Around Production Ready Web Apps with Docker Compose
- Avoiding 2 Compose Files for Dev and Prod with an Override File
- Reducing Service Duplication with Aliases and Anchors
- Defining your
HEALTHCHECKin Docker Compose not your Dockerfile - Making the most of environment variables
- Using Multi-stage builds to optimize image size
- Running your container as a non-root user
- Docker Best Practices for Python Developers
- Use multi-stage builds
- Pay close attention to the order of your Dockerfile commands to leverage layer caching
- Smaller Docker images are more modular and secure (watch out for Alpine though)
- Minimize the number of layers (
RUN,COPY,ADD) - Use unprivileged containers
- Prefer
COPYoverADD - Cache python packages to the docker host
- Prefer array over string syntax
- Understand the difference between
ENTRYPOINTandCMD - Include a
HEALTHCHECKinstruction - Whenever possible, avoid using the
latesttag. - Don't store secrets in images
- Use a
.dockerignorefile (include**/.git, etc.) - Lint and Scan Your Dockerfiles and Images (e.g. with
hadolint) - Log to stdout or stderr
- Docker Containers Security
Documentation
- Documentation-Driven Development
- Writing automated tests for your documentation: this should be required, IMO. Testing code samples in your documentation ensures they never get outdated.
- 🏙 Documentation is king, Kenneth Reitz
- Keep a Changelog
- Architectural Decision Records (ADR): a way to document architecture decision.
- Documenting Architecture Decisions
- [joelparkerhenderson