Facebook created a tool which detect and repair bugs automatically
Discover more from Tech World With Milan Newsletter
Facebook created a tool which detect and repair bugs automatically
This week’s issue brings to you the following:
Facebook created a tool which detect and repair bugs automatically
How Discord Stores Trillions of Messages
What is the Twelve-Factor App?
Bonus: The Startup CTO’s Handbook
So, let’s dive in.
Postman's VS Code Extension (Sponsored)
Postbot is now available across Postman with enhanced capabilities! The latest refresh of Postbot now offers a consistent, conversational interface available to you across your workspace. Learn how you can leverage Postbot throughout Postman to get contextual assistance.
The recent document by engineers from Facebook explains how they wrote a tool that can automatically fix bugs. In the paper, they introduced SAPFIX, an automated tool designed to detect and repair bugs in software. The tool has suggested fixes for six essential Android apps in the Facebook App Family: Facebook, Messenger, Instagram, FBLite, Workplace, and Workchat.
How Does It Work?
Step 1: Detect a Crash - Another tool, Sapienz, finds app crashes. When Sapienz identifies a crash, it is logged into a database.
Step 2: Identify the Problem - SAPFIX pinpoints the exact line of code causing the issue. It first checks if the crash is reproducible. If it's not reproducible, the crash is discarded. It uses a technique called "spectrum-based fault localization" to identify the most likely lines of code responsible for the crash.
Step 3: Suggest a Fix - Using predefined templates or code mutations, SAPFIX proposes a solution. After identifying the fault location, SAPFIX attempts to generate a patch. It employs two strategies:
Template-based Fixing: SAPFIX uses predefined templates to suggest fixes for common bugs. These templates are designed based on standard developer practices.
Mutation-based Fixing: SAPFIX resorts to a mutation-based system if the template-based approach fails. It systematically applies a series of code mutations to the fault location to generate potential fixes.
Step 4: Test the Fix - The proposed solution is tested to ensure it's valid. It uses the test cases from Sapienz to check the validity of the patch. If the patch passes all tests, it's considered a good fix. After patch validation, SAPFIX uses Infer (a static analysis tool) to analyze the proposed fix further. Infer checks if the patch introduces any new potential issues.
Step 5: Review - Developers get the final say, reviewing and approving the fix.
To learn more about it, check the entire document.
How Discord Stores Trillions of Messages
Discord engineers recently posted a description of their billion-message message storage system. They got underway in 2017. when they migrated from MongoDB to 12 Cassandra nodes to store billions of messages, they had different problems there, which were unpredictable as a server with small groups of friends tended to send a lot fewer messages than a server with hundreds of thousands of people. Additionally, they discovered that because reads in Cassandra require querying the memtable and on-disk files, they are far more expensive than writes.
Their solution was to migrate to ScyllaDB, a Cassandra-compatible database written in C++, which promises better performance and garbage collection. It is an open-source distributed NoSQL wide-column data store compatible with Apache Cassandra.
They started small but then decided to move all their databases. Nevertheless, hot partitions—many concurrent reads on a server that cause latencies—persisted despite all of this. They attempted to address the issue by using data services, which are positioned in between a ScyllaDB database cluster and their API monolith.
For writing data services, they used Rust, which gave them a C/C++ speed with thread and memory safety. In addition, these data services contain one gRPC endpoint per DB query, and if more users query the same data, the DB will be queried only once.
In the end, this enabled that if someone sent a message on a large server to everyone, the DB would not get traffic spikes anymore. Their tail latencies improved significantly, between 40-125ms on Cassandra, with ScyllaDb to 15ms.
What is the Twelve-Factor App?
It describes many well-tested architectural patterns and best practices for software-as-a-service (SaaS) applications. When apps are deployed to the web, they can be created with portability and resilience thanks to these best practices.
It was created by engineers at Heroku, a cloud platform-as-a-service company, around 2011. Adam Wiggins, a co-founder of Heroku, played a significant role in articulating and promoting these principles. This framework continues to be influential, aiding in the design and deployment of scalable and maintainable software applications.
The Twelve Factors:
Codebase - One codebase tracked in revision control, many deploys
Dependencies - Explicitly declare and isolate dependencies
Config - Store config in the environment
Backing services - Treat backing services as attached resources
Build, release, run - Strictly separate build and run stages
Processes - Execute the app as one or more stateless processes
Port binding - Export services via port binding
Concurrency - Scale out via the process model
Disposability - Maximize robustness with fast startup and graceful shutdown
Dev/prod parity - Keep development, staging, and production as similar as possible
Logs - Treat logs as event streams
Admin process - Run admin/management tasks as one-off processes
Beyond the twelve-factor application
Also, in the book "Beyond the 12 Factor App", Kevin Hoffman presents new guidelines that build on the original 12 factors but include elements such as telemetry and security:
One codebase, one application - Applications and repositories should be one-to-one.
API first - Make your API a first-class citizen in your development process.
Dependency management - Define dependency for each application.
Design, build, release, and run - We must define our development process in these four steps and execute them independently.
Configuration, credentials, and code - Manage configuration, credentials, and code separately.
Logs - We want to treat logs as event streams and use tools to analyze them.
Disposability - Make the possibility to restart your apps quickly.
Backing services - Define backend services and connect them via Cloud functions.
Environment parity - All environments should be the same.
Administrative processes - Things like DB migrations or batched jobs. Run them as a separate function from the application.
Port binding - Mapping application ports to actual ports on the Cloud infrastructure.
Stateless processes - We want the possibility to restart the server quickly at any time.
Concurrency - Design your applications so they can handle multiple processes.
Telemetry - Create a way to measure the application's performance (logs, domain telemetry, APM, etc.).
Authentication and authorization - Implement security from the beginning of your project.
Also, check IBM's view on this topic: “Beyond the 12 factors: 15-factor cloud-native Java applications”.
Bonus: The Startup CTO’s Handbook
Check this useful (free) handbook, created by Zach Goldberg, which covers some essential topics for all tech leaders, such as:
Leadership
Engineering management
Different tech topics (architecture, Testing, DevOps, etc.)
More ways I can help you
1:1 Coaching: Book a working session with me. 1:1 coaching is available for personal and organizational/team growth topics. I help you become a high-performing leader 🚀.
Promote yourself to 18,000+ subscribers by sponsoring this newsletter.