System design (not confused with both design system and design pattern) is way too complicated to teach properly in university because it touches too many concepts, technologies, softwares, and patterns. It is something expected to pick up eventually along your career.
I’ve seen far too often that many field experts, project managers, startup (co)founders, or developers with almost zero experience in system design were put in charge of designing one.
Software architect is the only person you can assume to be good at system design because it is the heart of this job. Everyone else is just a hit and miss, the worst is when nobody in charge.
Don’t assume that you can design a good system when you’re good at coding and not learning properly.
System design is the process of defining the elements of a system such as the architecture, modules and components, the different interfaces of those components and the data that goes through that system.
System design is a very broad and open-ended topic. Even a software engineer with many years of working experience at a top IT company may not be an expert on this field.
System design is not as simple as following some specifications when coding, it’s about following the right patterns with enough adjustments at the right time and at the right place.
System design is hard and often shared by experienced engineers who have extensive experiences working with big systems, don’t expect to read anything good online as blog posts, paid books are the go-to resources.
If you want to become an expert, you need to read many books, articles, and solve real large scale system design problems. How good is good enough? People always start an answer with “It depends”.
Most engineers struggle with the system design partly because of their lack of experience in developing large-scale systems and partly because of the unstructured nature. Even engineers who’ve some experience building such systems aren’t comfortable with these tasks, mainly due to the open-ended nature of design problems that don’t have standard solutions.
To design a good system you must make tons of technical decisions (languages, frameworks, logging, monitoring, testing, deployment, etc.) with business constraints (budget, human resource) then enforce your team to deliver that design in time.
Different types of software solutions — build something from scratch or buying cooked solutions, self-hosted services or cloud services, ultimate control and resource utilization — are something to keep in mind.
It’s both difficult and expensive to find an immediate software architect for upcoming project. Established software architects often work in enterprises and their solutions are mostly overkill with enterprise solutions.
What happens if nobody in charge? Eventually the system is a mess with tons of dump services which are hard to test and do nothing but CRUD, hard to automate from developer machines to production servers.
Internal employees are always preferred as a chance to grow, better understanding, faster adapting. That’s when someone in the team will take responsibility to design the system with naivety.
Naivety is the lack of experience!
What is the expectation? System design all is about following best practices with tradeoffs in mind. Anti-patterns not always bad, Patterns not always applicable. Do the right thing at the right time with reasonable complexity.
It’s good when you know a lot and do little. It’s worst when you’re trying to do a lot with little knowledge.
It’s possible to design a working system with little experience, the point is you must take responsibility and learn seriously onward.