Skip to content

Latest commit

 

History

History
1241 lines (1193 loc) · 65.6 KB

Backend Developement.md

File metadata and controls

1241 lines (1193 loc) · 65.6 KB
Raw

Backend Development

This can be found in roadmap form at NavigoLearn.com

Description

This is a roadmap that gives an overview of the key concepts and technologies you need to know to become a Backend Developer. It is not tied to a specific programming language which makes it a versatile guide.

Backend Development Intro: This section covers the basics. What is a server, a protocol and the internet works. It also discusses the roles, responsibilities, and common technologies that backend developers use.

Server-side Programming: Here you'll pick a programming language to work with and understand basic data structures of the chosen language.

Databases: You’ll learn about different types of DBMS's (Database Management Systems) like SQL, NoSQL and ORMs for database interaction.

APIs: This section discusses the different types of APIs such as RESTful API, GraphQL API, etc.

Authentication & Authorization: It covers the theories of authentication and authorization as well as various techniques such as Session Authentication, JSON Web Tokens (JWT), OAuth, and OpenID Connect.

Caching: Here you'll learn about caching to enhance performance.

Containerization: Covers technologies like Docker, Kubernetes for containerizing applications making them scalable and manageable.

Cloud Providers: Provides an overview of major cloud service providers.

Backend Architectures: Covers the popular architectural models used in backend development.

Testing: Discusses various types of testing as well as popular testing frameworks, and various testing methodologies.

Logging and Monitoring: Covers various aspects of monitoring an application from log management and network monitoring to performance and security monitoring.

CI/CD: Covers the basics of Continuous Integration and Continuous Deployment, including the best practices and tools for both.

Rate Limiting and Throttling: Discusses the techniques and libraries for rate limiting and throttling to ensure fair usage of limited resources.

This path should give you a broad understanding of backend development. Please remember this is a guide and not a strict rulebook. Backend development is vast, and this roadmap provides a way to navigate through resources available to the best of author's knowledge. Be sure to take each section at your own pace, even out of order if you wish. Ensuring you understand each concept thoroughly.

Table of contents

  1. Backend Development Intro
    1. What is a server?
    2. What is a protocol?
    3. How the internet works?
    4. Pentagon Backend (What/When/Where/Why/How)
      1. Role of a Backend Developer
      2. Common Backend Technologies
  2. Server-side Programming
    1. Choose a language
    2. Basic Data Structures
  3. Databases
    1. SQL
      1. SQL Injections
    2. NoSQL
      1. MongoDB
      2. Redis
      3. Cassandra
    3. ORMs
  4. APIs (Application Programming Interfaces)
    1. Types of APIs
      1. RESTful APIs
        1. HTTP Methods
        2. RESTful API Design
        3. RESTful API Best Practices
      2. GraphQL APIs
      3. ProtoBuf
    2. Websockets & Real-time Communication (WS/WSS)
    3. API Security
  5. Authentication & Authorization
    1. Understanding Authentication
    2. Understanding Authorization
    3. Authentication Techniques
      1. Session Authentication
      2. JSON Web Tokens (JWT)
      3. OAuth
      4. OpenID Connect
  6. Caching
    1. Types of caching
      1. In-Memory Caching
      2. HTTP Caching
      3. Database Query Caching
    2. Cache Invalidation
    3. Cache Best Practices
  7. Containerization
    1. Docker
      1. Kubernetes
      2. Docker Swarm
      3. Apache Mesos
      4. Helm
    2. LXD
    3. Podman
    4. Containerd
  8. Cloud Providers
  9. Backend Architectures
    1. Monolithic
    2. Microservices
    3. Serverless
    4. Service-Oriented Architecture (SOA)
  10. Testing
    1. Types of testing
    2. Testing frameworks
      1. Node.js
      2. Python
      3. Go
      4. PHP
      5. Kotlin
    3. Testing Methodologies
      1. Test Driven Development (TDD)
      2. Behavior Driven Development (BDD)
      3. Exploratory Testing
      4. Stress Testing
      5. Security Testing
  11. Logging and Monitoring
    1. Application Monitoring
      1. Log Management
      2. Infrastructure Monitoring
      3. Network Monitoring
      4. Security Monitoring
      5. Performance Monitoring
    2. Tools for Logging and Monitoring
  12. CI/CD (Continuous Integration / Continuous Deployment)
    1. Basics of CI (Continuous Integration)
    2. Tools for CI
      1. Jenkins
      2. TravisCI
      3. CircleCI
      4. GitHub Actions
    3. Basics of CD (Continuous Deployment)
    4. Tools for CD
      1. Jenkins
      2. AWS CodeDeploy
      3. Google Cloud Deployment Manager
      4. Heroku
    5. CI/CD Best Practices
  13. Rate Limiting and Throttling
    1. Rate Limiting Techniques
      1. Fixed Window Counter
      2. Sliding Window Log
      3. Token Bucket
      4. Leaky Bucket
    2. Libraries for Rate Limiting
      1. Node.js
      2. Python
      3. Go
      4. Kotlin

Essential Prerequisites

  • Basics of computer science
    • You should have a basic understanding of how computers work. This includes things like how memory works, how the CPU works, etc. You don't need to know everything, but you should have a basic understanding of how computers work.

Recommended Prerequisites

  • Git and GitHub
    • For collaboration and version control and to get familiar with the development workflow.
  • Data Structures and Algorithms
    • I go into this a bit in this roadmap.
    • Understanding of data structures and algorithms is essential for backend development. You don't need to know every algorithm under the sun, but you should have a basic algorithmic foundation.
  • Servers and HTTP/HTTPS
    • I go into this a bit in this roadmap.
    • It's important to know a bit about how http works and how servers work. This will help you understand how to build applications that work on the internet.
  • Basic frontend technologies
    • Technologies such as HTML, CSS and a little bit of JavaScript
    • You don't need to be an expert, you need to know this to properly communicate with frontend developers and to understand how the frontend and backend work together.

  1. Backend Development Intro

    • This section contains information on what you need to know to get started with backend development. You don't need to know everything in this subtree to get started, but it's important to have a basic understanding of the concepts listed here, so you can talk to people and get help if you get stuck working on a project.
    • Learn web development as an absolute beginner - Coder Coder
    • 1.A What is a server?
      • It's important to understand the concept of what a server is and how it works. This will help you understand how anything on the internet works. You may already have a vague idea if you're a gamer, but could you really define what a server is?
      • What is a Server? (Deepdive) - Video - LiveOverflow
      • What is a Server? - Article - GeekforGeeks
    • 1.B What is a protocol?
    • 1.C How the internet works?
    • 1.D Pentagon Backend (What/When/Where/Why/How)
      • Why do you want to learn backend development? What value does it add? What problems does it solve? These are all questions you should ask yourself before you start learning backend development.
      • 1.DA Role of a Backend Developer
      • 1.DB Common Backend Technologies
        • What technologies do backend developers use? What skills do they need?
        • A list of things that are commonly used in backend development. And most of what you'll be learning in this roadmap.
        • Languages & Frameworks
          • Be it Express, Django or any other framework, the easiest way to get started with backend development is to use a framework. Frameworks are a collection of libraries and tools that make it easier to build applications by abstracting away the complexity of the underlying technologies.
        • Databases & ORMs
          • Databases are used to store data. There are many different types of databases, but the most common ones are relational databases like MySQL, PostgreSQL, etc. and NoSQL databases like MongoDB, Redis, etc. ORMs are used to interact with databases. They allow you to write code instead of SQL queries to interact with the database.
        • APIs & communication Protocols
          • APIs are used to communicate between different applications. They allow you to send and receive data between applications. There are many different types of APIs, but the most common ones are REST APIs, GraphQL APIs, and WebSockets.
        • Caching
          • Caching is used to store data in memory so that it can be accessed faster. It's used to improve the performance of applications by reducing the number of times data needs to be fetched from the database or other external sources.
        • Identity & Access
          • Authentication is used to verify that a user is who they say they are. Authorization is used to determine what a user can and cannot do. This can be done using a variety of different methods, but the most common ones are JWTs and OAuth.
        • Containerization
          • Containerization is used to package applications into containers so that they can be run on any machine without having to worry about dependencies or other external factors. The most common containerization tool is Docker.
        • Cloud Providers
          • Cloud providers are used to host applications in the cloud. They provide a variety of different services, but the most common ones are storage, compute, and networking. The most popular cloud providers are AWS, Azure, GCP and DigitalOcean.

  2. Server-side Programming

  3. Databases

  4. APIs (Application Programming Interfaces)

  5. Authentication & Authorization

Checkpoint: You should now have a basic understanding of backend - Front-end developer level

You should be able to use backend APIs at this point to integrate them into your frontend applications.

You should also be able to build simple backend applications.

Project ideas:

  • To Do List app (REST API)
    1. Add Task
    2. Edit Task
    3. View Task
    4. Delete Task
    5. Mark Task as Done
    6. What else can you think of?
  • Simple Blog app (REST API)
    1. Add Post
    2. Edit Post
    3. Fetch Post
    4. Delete Post
    5. Search Post? — Database focused task
    6. What else can you think of?
  • Simple Chat App (WebSockets + REST)
    1. Send Message
    2. Receive Messages
    3. Delete Message
    4. Search Message history? — Database focused task
    5. What else can you think of?
  • E-commerce App (REST API)
    1. Add Product
    2. Edit Product
    3. Fetch Product
    4. Delete Product
    5. Search Product? — Database focused task
    6. What else can you think of?

  1. Caching -

    • Fair warning this section has a lot of text articles and not many videos.
    • Caching is the process of storing copies of files or data in a cache, or a temporary storage location, so that they can be accessed more quickly. Databases, web pages, and other objects can be cached in response to repeated requests for the same data.
    • Caching - Simply Explained - Video - Simply Explained
    • Caching Overview - Article - Amazon Web Services
    • 6.A Types of caching
      • There are several types of caching that you could use depending on your needs, such as in-memory caching (like Memcached or Redis), HTTP caching, database query caching, etc.
        • 6.AA In-Memory Caching
          • Here is where you store data in memory, so that it can be accessed faster. It's used to improve the performance of applications by reducing the number of times data needs to be fetched from the database or other external sources. We talked about this in the Redis node. Another example of in-memory caching is Memcached.
          • Cache Memory Explained - Video - ALL ABOUT ELECTRONICS
          • Redis Caching in Node.js - Video - Traversy Media
        • 6.AB HTTP Caching
          • HTTP caching refers to the practice of storing copies of web resources in a cache close to the user for faster access. The browser storage, service workers and proxies can serve as HTTP caches. This improves the performance and efficiency of web applications by reducing the load on the server.
          • Caching - Article - Google Developers
          • HTTP Caching - Article - MDN Web Docs
        • 6.AC Database Query Caching
    • 6.B Cache Invalidation
    • 6.C Caching Best Practices
      • Caching needs to be implemented carefully as improper caching can lead to users being served stale or inappropriate data. There are several best practices to follow when implementing caching.
      • Principles of Cache Design in Web Application - Article - Philip Feng Ph.D. on Medium

  2. Containerization

    • Containerization is a method of using operating systems' virtualization features to run applications and services in resource-isolated processes. Containers bundle an application together with all related configurations, libraries, and files necessary to run. This ensures the application works in any environment. Docker is one of the most popular tools for creating and managing containers.
    • Never install locally - Video - Coderized
    • 7 Cases When You Should Not Use Docker - Article - freeCodeCamp.org
    • 7.A Docker
    • 7.B LXD
      • LXD is a next generation system container manager. It offers a user experience similar to virtual machines but using Linux containers instead. It's image based with pre-made images available for a wide number of Linux distributions and is built around a very powerful, yet pretty simple, REST API.
      • LXD - Documentation - Ubuntu (Canonical parent company of both projects)
    • 7.C Podman
      • Podman is a daemon-less container engine for developing, managing, and running Containers on Linux. The design of Podman is to be a drop-in replacement for Docker. Podman uses the same command-line interface as Docker, however, it operates without a daemon, and it employs container runtimes like runc for running containers.
      • Get Started with Podman - Documentation - Podman
    • 7.D Containerd
      • Containerd is an industry-standard core container runtime. It is available as a daemon for Linux and Windows, and it manages the complete container lifecycle of its host system, which includes image transfer and storage, container execution and supervision, and low-level storage handling. Containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users.
      • Getting started with containerd - Documentation - Containerd

  3. Cloud Providers - Cloud providers are platforms offering a range of computing services over the internet. These services typically include servers, storage, databases, networking, software, analytics, and even artificial intelligence. Major cloud providers include Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure. They enable businesses to run their applications without needing to maintain the underlying infrastructure, offering scalability and flexibility.

  4. Backend Architectures

  5. Testing

Checkpoint: You should now have a general understanding of backend development. You should be able to build more advanced WebApps and APIs with this knowledge.

Project ideas: Sorry I got none... you can suggest some in the issues section on GitHub. (https://github.com/NavigoLearn/RoadmapsMarkdown)

  1. Logging and Monitoring

    • Logging and monitoring are essential for any application. They allow you to see what's happening in your application and how it's performing. They also allow you to see if there are any errors or issues that need to be fixed. You should learn your language's logging and monitoring tools.
    • JavaScript — Beyond Console Log in 100 Seconds - Video - Fireship
    • Python — Python Logging: How to Write Logs Like a Pro! - Video - ArjanCodes
    • Go — How to Properly Log Things in Go (Native) - STOP using fmt.Println() - Video - GoLang Dojo
    • PHP - How to Log Errors in PHP | PHP Error Reporting and Debugging for Beginners | Log Website Errors - Video - Dani Krossing
    • 11.A Application Monitoring
      • Application monitoring is a process that ensures that a software application processes and performs tasks as expected. The purpose of monitoring is to detect and resolve any issues or abnormalities that could affect expected and efficient operation.
      • 11.AA Log Management
        • Log management includes all the stages from the creation of log events to their ultimate disposal. Proper log management practices can give valuable insights into system performance, security, user behaviors, system functionality, and troubleshooting issues.
        • What is Log Management? - Article - SolarWinds
      • 11.AB Infrastructure Monitoring
        • Infrastructure monitoring is the task of ensuring that all the hardware, software, networks, and processes in your IT infrastructure are running smoothly and optimally.
        • Infrastructure Monitoring Overview - Article - DataDog
      • 11.AC Network Monitoring
        • Network monitoring is the use of a system that constantly monitors a computer network for slow or failing components, and that notifies the network administrator (via email, SMS or other alarms) in case of outages or other trouble.
        • What is Network Monitoring? - Article - IBM
      • 11.AD Security Monitoring
        • Security monitoring is the collection, analysis, and escalation of indications and warnings to detect and respond to intrusions. This is crucial to ensure the safety of your applications and infrastructure.
        • What is Security Monitoring? - Article - Hewlett Packard Enterprise
      • 11.AE Performance Monitoring
        • Performance monitoring is the process of detecting and diagnosing system performance problems. It helps you maintain system efficiency, meet service-level agreements (SLAs), and identify areas of improvement.
        • Performance Monitoring - an overview - Article - Science Direct
    • 11.B Tools for Logging and Monitoring
      • There are many tools available for logging and monitoring your systems. Here are a few of the most popular:
      • Loggly - Product - Cloud-based log management and analytics service that leverages machine-generated big data to deliver real-time IT insights.
      • Splunk - Product - Software for searching, monitoring, and analyzing machine-generated big data, via a web-style interface.
      • Elasticsearch - Product - A search and analytics engine based on the Lucene library.
      • Logstash - Product - An open-source data collection engine with real-time pipelining capabilities.
      • Kibana - Product - An open-source data visualization plugin for Elasticsearch.
      • Prometheus - Product - An open-source systems monitoring and alerting toolkit.
      • Grafana - Product - An open platform for beautiful analytics and monitoring.
      • New Relic - Product - A cloud-based platform that gives devs, ops, and tech teams the tools to monitor and optimize their operations
      • DataDog - Product - Monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform.
      • Sentry - Product - Open-source error tracking that helps developers monitor and fix crashes in real time.

  2. CI/CD (Continuous Integration / Continuous Deployment)

  3. Rate Limiting and Throttling

Checkpoint: You should now have a broad understanding of everything going into backend development. You should be able to build about anything you want with this knowledge.

Project ideas:

  • Build a script scheduler
    1. Develop a functionality that permits users to create scripts running on a schedule that they specify.
    2. Implement a feature to allow users to select the script interpreter (like Node.js, Python, Bash, Zsh, Lua, etc.) to use based on their script's requirements.
    3. Allow users to set webhooks that become active when the script finishes its run.
    4. Provide the ability for users to establish environment variables specific to the script.
    5. Build a permissions system and restrict certain commands, making them accessible only with additional permissions.
    6. Engineer a simple frontend application to manage the scripts.
    7. Incorporate an interface to view the console output of the executed scripts.
    8. Log the output of the scripts either in a database or a log file for debugging purposes.
    9. Include a feature to view the logs of the scripts in the web user interface, provided the user has necessary permissions.
    10. Introduce an option for users to pause, stop, or resume scripts.
    11. Add user authentication and authorization to ensure safe access to the scheduler.
  • Build a web crawler
    1. Establish your target website(s) for crawling.
    2. Build a parser to extract information.
    3. Store this extracted information in a structured format (like JSON, XML or in a Database).
    4. Implement an approach to navigate through different web pages and links.
    5. Reliably handle runtime exceptions and errors.
    6. Design a system to respect robots.txt rules.
    7. Add user-agent strings and manage request headers.
    8. Introduce multi-threading or asynchronous processing to speed up crawling.
    9. Create a result viewer or output handler to inspect, validate and use your crawling data.