Bytebase has obtained SOC 2 Type 2 certification.

SOC 2 Type 2 Certification

Bytebase has obtained SOC 2 Type 1 certification, which is a significant milestone for Bytebase's security and compliance efforts.

SOC 2 Type 1 Certification

Database DevSecOps, maintain developer velocity without compromising data security

Bytebase 3.0 - DevSecOps for Database

Bytebase just passed 10,000 stars on GitHub, thanks to our global community.

Bytebase Surpasses 10,000 Stars on GitHub

Bytebase is pleased to announce a partnership with Vettabase, an open source database services company with a focus on database automation.

Bytebase Has Partnered with Vettabase!

As Q3 2023 comes to an end, let's take a look back at what we've accomplished at Bytebase in the past three months.

Bytebase Q3 2023 in Retrospect

Join us at the 10th year of Hacktoberfest celebrating open-source!

Hacktoberfest 2023 with Bytebase

We are proud to announce the Technology Partnership with GitLab to provide developers with a smooth database development experience.

Announcing Partnership with GitLab to Automate Database Development

Bytebase just passed 1 million downloads, thanks to our customers and community users all over the world.

Bytebase Crossed 1 Million Downloads

Pioneering Database Change Management and bring the GitLab/GitHub experience to database development

Bytebase 2.0 - the GitLab for Database DevOps

As Q1 2023 comes to an end, let's take a look back at what we've accomplished at Bytebase in the past three months.

Bytebase Q1 2023 in Retrospect

Announcing SQL Chat, chat-based SQL Client for querying data and answering DB questions.

SQL Chat - Chat with Your Database

This article demonstrates the Single Sign-On (SSO) protocols supported by Bytebase. SSO allows users to log in once and access all authorized resources, improving security, user experience, and organizational flexibility.

Introducing Single Sign-On in Bytebase

We are proud to announce the Technology Partnership with GitHub to provide developers with a smooth database development experience.

Announcing Partnership with GitHub to Automate Database Development

We are proud to announce our partnership with PingCAP, which will help developers and DBAs to more efficiently and safely manage the lifecycle of application database schemas.

Streamline Database Change Management for TiDB Cloud with Bytebase

With Terraform Bytebase Provider, teams can codify Bytebase resource configurations, including environments, database instances, and roles via Terraform.

Introducing Terraform Bytebase Provider

Bytebase 1.11.0 brings MongoDB support. This blog post walks you through managing MongoDB with Bytebase.

Introducing MongoDB Support in Bytebase

Introduce the new SQL Editor in Bytebase Console with six highlights - Admin mode, database-based access control, data masking, audit logs, collaborative sheets and easy to use.

The SQL Editor for Developers and DBAs

DB Cost is a tool for developers to compare performance and cost differences among cloud database providers.

Introducing DB Cost, the simple cloud database pricing sheet for AWS RDS and Google Cloud SQL instance

Bytebase is now the first solution to be included in the CNCF Landscape that focuses solely on database CI/CD.

CNCF Landscape includes Bytebase as the first-ever database CI/CD solution

Bytebase is participating in Hacktoberfest 2022! Join the month long celebration of open source with us.

Hacktoberfest 2022 with Bytebase

Introduce gh-ost based online schema migration and how to perform an online schema migration without downtime in Bytebase.

gh-ost Based Online Schema Migration for MySQL

This post introduces a new feature - schema review policy, which help DBAs and application developers compose better schema change statements.

Introducing Schema Review Policy for MySQL

Bytebase is a tool for database schema change and version control, its main audience are Developers and DBAs. We want to make Bytebase the industry benchmark solution for managing database schema change.

Announcing Bytebase. Open source, web-based, zero-config, dependency-free database schema change and version control tool for Developer and DBA


Bytebase is an open-source database DevOps tool; it’s the GitLab for managing databases throughout the software development lifecycle (SDLC). It offers a web-based workspace for Developers and DBAs to collaborate and execute database changes safely and efficiently.

Database schema migration is one of the most common operations for developers and DBAs. However, it is one of the most challenging operations as well. What’s more painful, it will take hours, even days to migrate schema for large tables. Regular schema migration must lock the database table and render the service offline. For most businesses, it’s unacceptable to endure such extended downtime. To solve it, the industry has developed various online schema change solutions:

- [gh-ost](https://github.com/github/gh-ost)
- [Pt-online-schema-change](https://www.percona.com/doc/percona-toolkit/3.0/pt-online-schema-change.html)
- [Facebook OSC](https://github.com/facebookincubator/OnlineSchemaChange)

By adopting these tools, teams can reduce schema migration downtime from days and hours to seconds. On the other hand, operating these tools still requires a lot of expertise. And their usage is limited to the seasoned DBAs, while Developers still need to ask their DBA peers to perform online schema changes on their behalf.

Since Bytebase is a database tool for the DevOps teams, we want to hand over the powerful online schema change capability to the Developers.

In this blog post, we will introduce you to the beta release of Bytebase online schema migration solution based on gh-ost. Bytebase solution provides a step-by-step interface with continuous progress updates to oversee the entire migration process.

## Why We Choose gh-ost

All existing online schema change solutions share a similar workflow:
![online-schema-migration](/content/blog/gh-ost-based-online-schema-migration-for-mysql/online-schema-migration.webp)

1. Create a new, empty table (called ghost/shadow table) with an exact copy of the original table’s schema
2. Apply schema migration to the ghost table
3. Copy the existing rows from the original table to the ghost table
4. Capture the ongoing changes to the original table and continuously apply them to the ghost table
5. Swap the original table with the ghost table

Why do we decide to integrate gh-ost into Bytebase? The most crucial factor is that gh-ost is a triggerless online schema migration solution for MySQL. gh-ost uses the binary log stream to capture table changes and asynchronously applies them to the ghost table. As a result, it avoids the performance overhead due to trigger and lock contention. Secondly, gh-ost has greater control over the migration process. Bytebase can leverage it to expose the migration progress and provide a better user experience with parameters supported by gh-ost.

## How to Make Online Schema Migration for Large Tables

Bytebase provides two types of schema migration: regular migration and gh-ost. You can operate it by creating an issue in three steps. Let’s demonstrate how to use gh-ost to alter schema.

The screenshot below shows the table “sbtest1” information before applying schema migration. We can see that the table has 4 columns and 694 MB data.
![overview](/content/blog/gh-ost-based-online-schema-migration-for-mysql/overview.webp)

**Step 1**. Create a gh-ost issue

1. Click **Alter Schema** on the database details page.
   ![step_1_1](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_1_1.webp)

2. Select **Online migration** and click **Next.**
   ![step_1_2](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_1_2.webp)

3. Enter SQL statements for the task “Sync data”, and click **Create** after selecting a reviewer to approve the issue.
   ![step_1_3](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_1_3.webp)

**Step 2**. Execute the task “Sync data”.
The task “Sync data” runs after the designated reviewer clicks **Approve**. See the screenshot below, it shows that the progress of “Sync data” is 59%.
![step_3](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_3.webp)

**Step 3**. Perform the task “Switch tables”.
The screenshot below shows that the first task has been completed.  
![step_4_1](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_4_1.webp)

Then the designated reviewer clicks **Approve** to execute the second task “Switch tables”.
![step_4_2](/content/blog/gh-ost-based-online-schema-migration-for-mysql/step_4_2.webp)

Here is the result of gh-ost. It says that the database performs gh-ost successfully.
![result](/content/blog/gh-ost-based-online-schema-migration-for-mysql/result.webp)

## Learn More

Bytebase supports online schema migration for large tables based on gh-ost, a triggerless online schema migration solution that can reduce the application downtime from hours to seconds. It offers a step-by-step interface with continuous progress update to oversee the entire migration process. You can check out the [documentation](https://docs.bytebase.com/change-database/online-schema-migration-for-mysql) to learn more.

If you have any feedback, we’d love to hear from you. You can find us on our [GitHub issue board](https://github.com/bytebase/bytebase/issues).



Bytebase is an open-source database DevOps tool, and it’s the GitLab for managing databases throughout the application development lifecycle. It offers a web-based workspace for DBAs and Developers to collaborate and manage the database change safely and efficiently.

The trend of codifying database schema changes (aka Database as Code) has accelerated in recent years. Today, there exists a bunch of tools to check the code quality; however, the ability to check SQL statement quality is quite lacking.

As the trailblazer to embrace Database as Code, Bytebase aims to lift the developer experience of working with SQL. Today we are introducing a new feature - schema review policy, a set of SQL quality checks to help DBAs and application developers compose better schema change statements.

## What is Schema Review Policy?

A schema review policy is a set of SQL-style rules to check the SQL statement quality. DBAs can configure schema review policies for each environment. Bytebase will check SQL statements automatically before executing schema changes and block them if any configured policies are not met.

The schema review policy is available to MySQL and TiDB. PostgreSQL support will be coming soon.

## Rules in Schema Review Policy

This initial launch includes 15 rules across 6 categories.

### Naming conventions

- Table naming check
- Column naming check
- Unique key naming check
- Index naming check
- Foreign key naming check

### Statement

- Disallow “SELECT \*”
- Require “WHERE”
- Disallow leading wildcard % in LIKE

There are countless stories about the performance issue and database outages caused by bad SQL statements. Bytebase provides these rules to avoid the common problem.

### Table

- Require primary key

It is hard to identify a particular record for a table without a primary key. Bytebase provides the rule - “Require primary key” to enforce each table to have a primary key.

### Schema

- Backward compatibility check

Introducing backward-incompatible schema changes is one of the most common mistakes made by developers. And enforcing backward-compatible schema change is the standard practice many engineering organizations adopt. Bytebase provides the built-in backward compatible check to catch all common incompatible schema change scenarios.

### Column

- Required columns: make sure all tables have the required columns
- Disallow NULL: all columns can not have a NULL value

### Engine

- Require InnoDB

InnoDB is the default storage engine of MySQL 5.5+ with powerful transaction features.

## How to Configure Schema Review Policy

Bytebase provides a schema review policy template based on the industry best practices. You can apply it to your production environment directly. Of course, you can create your own schema review policy based on it.

Let’s begin configuring a policy from the environment dashboard.

**Step 1**. Select the tab “Prod” on the environment dashboard, and click the button “Configure policy” at the bottom of the page.

![step1](/content/blog/schema-review-policy-mysql/schema-review-policy-step1.webp)

**Step 2**. Select the template “Production Environment Template” and click the button “Next”.

![step2](/content/blog/schema-review-policy-mysql/schema-review-policy-step2.webp)

**Step 3**. Modify the error-level and rule-specific fields for rules, then click the button “Next” to preview your schema review policy.

![step3](/content/blog/schema-review-policy-mysql/schema-review-policy-step3.webp)

**Step 4**. Click the button “Confirm and add” to apply your policy to the environment.

![step4](/content/blog/schema-review-policy-mysql/schema-review-policy-step4.webp)

Now you can see the schema review policy that you jsut created on the page below.

![dashboard](/content/blog/schema-review-policy-mysql/schema-review-policy-dashboard.webp)

## How Schema Review Policy is Used

When you create an issue in an environment configured with a schema review policy, Bytebase automatically checks SQL statements and shows the schema review result. Take the screenshot below as an example. It shows that the input SQL statement does not meet the configured policies.

![issue](/content/blog/schema-review-policy-mysql/schema-review-policy-issue.webp)

You can get more details by clicking the button “SQL review”.

![details](/content/blog/schema-review-policy-mysql/schema-review-policy-details.webp)

Please try it out. If you want to suggest additional rules, please [let us know](https://github.com/bytebase/bytebase/issues) .



After 6 months of intense development and over 2,000 commits, Bytebase version 0.1.0 finally arrives. The code is open sourced at [https://github.com/bytebase/bytebase](https://github.com/bytebase/bytebase) using Apache 2.0.

## **What is Bytebase**

Bytebase is a tool for database schema change and version control, its main audience are Developers and DBAs.

Database schema migrations (aka database migrations), like code changes, are one of the routine tasks for developing applications. Although schema changes are not as frequent as code changes, they are much more dangerous. Since the data is stateful, unlike the fast code rollback, the database rollback is much more complicated. Besides database changes, network changes are also dangerous and can cause large-scale service failures. While network configurations are usually stateless, so rollback and recovery are also faster.

> _Thus, database changes are usually the most likely ones causing long-lasting large-scale service outages. And in the extreme case, can even bankrupt the entire business._

Since this operation is so critical and impactful, there must be many tools to solve this problem? Yes, but the reality is although the database industry has more than 40 years history. DevOps, Database CI and even Database-as-Code have been hovering around for many years. There is still no industry benchmark solution handling this task, like GitLab to code hosting, Prometheus to service monitoring.

## **Why we build Bytebase**

Put simply, it's to answer the previous question

> _We want to make Bytebase the industry benchmark solution for managing database schema change._

Let's first evaluate the existing solutions first, roughly divided into 3 categories:

1. Tools been around for a long time

Many teams have already adopted tools like Flyway, Liquibase in their development workflow. These tools have already built a large customer base and a thriving community. They also have a solid schema change core and stood the test of time. But, these products are all more than 10 years old, and the tech stacks used are sort of dated. For example, Flyway and Liquibase are both based in Java. Nowadays, database changes are usually used as a step in CI, and the step is often executed in a separate container. To run a Flyway/Liquibase command, one must first download a JVM or use an image containing JVM. Also, they are thin on the frontend with no or basic functionalities. Another concern is they support everything from commercial to open source databases, regardless of XML or SQL migration format. This does carry weight over the years.

2. Homegrown tools

Usually when the dev group reaches ~50 people, a dedicated DBA must be summoned. And for a typical middle-sized to large company, the ratio of DBAs to Developers is between 1:150 to 1:400. To improve serving bandwidth, DBAs have to rely on tools. Starting from the most basic SQL review ticket, DBAs often would roll their own by using Python + Django. These tools are often stacked up bit by bit, starting with SQL review ticket, then cron inspector, and then monitoring, backup/restore, approval workflow etc, and finally we get a classic internal company system, like this:

![_](/content/blog/announce-bytebase/life-of-a-swe.webp)By Manu Cornet, [https://goomics.net/106/](https://goomics.net/106/)

3. Homegrown to open source tools

Some mindful engineers have open sourced their homegrown tools. A few have gained thousands of GitHub Stars, and been deployed in middle to large companies. These tools did provide great value. At least DBAs on longer need to reinvent wheels, especially those are not the wheels they are good at/enjoy building. But frankly speaking, from the underlying data model design all the way up to UI/UX, these open source tools are still quite far from the industry benchmark standard.

And we ask ourselves:

- What's the GitLab for database instead of for code?
- What's the Figma for DBAs instead of for designers?
- What's the Terraform for Database-as-Code instead of for Infrastructure-as-Code?

The current industry does not have a good answer yet. Before starting Bytebase, we used to be application developer, database kernel developer, DBA ourselves and have also managed the DevTools, DBA and even collaboration tool group in world's leading technology companies (Google and Ant Group). Thus we happen to have all the required skills to build such a tool.

## **How we build Bytebase**

Since this is a launch post, we will cover some design aspects, more to be covered in separate posts.

### Data modeling

Data modeling is always the center piece of the software, it's the one determining the celling of the software and is very hard to change after launch. Even being veterans in the database domain, we have spent tremendous time to make the data modeling right. Below is Bytebase's essence, its v1 data model:

![_](/content/blog/announce-bytebase/data-model.webp)

This is actually the 3rd iteration, and every iteration causes a significant refactoring. As an example, the upper right is to model the execution pipeline, Bytebase has studied all mainstream pipeline design and came up with a solution:

![_](/content/blog/announce-bytebase/data-model-pipeline.webp)

For those interested, please check our design notes: [https://github.com/bytebase/bytebase/tree/main/frontend/src/types/pipeline#readme](https://github.com/bytebase/bytebase/tree/main/frontend/src/types/pipeline#readme)

### UI and UX

To be honest, we lack the design skills to craft Bytebase to the level of the tools we aspire to like Figma, Notion, Linear. So our bottom line is to make it easy to use with a decent looking frontend. Looking at Bytebase's architecture graph:

![_](/content/blog/announce-bytebase/architecture.webp)

We deliberately put a large browser here to suggest the Bytebase console is a very critical piece of the product. Indeed, the frontend work takes 2/3 ~ 3/4 of the development time so far. And among those work, we spend most effort to polish the most frequently accessed UI - issue detail view.

### Issue detail view

![_](/content/blog/announce-bytebase/issue-detail-annotated.webp)

The number shows each sub-component of the issue detail view, let's take another look

![_](/content/blog/announce-bytebase/issue-detail-link.webp)

Each square shows an interactive area. Issue is the rendezvous, and we try to add link to every related object if possible.

### Slug design

Next let's talk about link, and in particular the slug part.

![_](/content/blog/announce-bytebase/slug-design.webp)

The URL is reached by first clicking the stage progress bar, followed by clicking the 2nd activity. This slug consists of 4 parts:

1. **add-createdat-column-to-userpostcomment-table-for-dev-environment**, the issue name.
2. **13008**, the issue ID and the key identifying the issue.
3. **?stage=dev-1**, query parameter, used to locate the stage, an issue can contain many stages and different stage can show different content.
4. **#activity14029**, anchor used to locate the exact comment position.

With this, user can learn the name, stage by skimming the URL usually seen in the doc, chat conversation. Also, by clicking the URL, user will navigate to the exact same place as the URL producer. All Bytebase pages have adopted this practice. BTW, you may notice we use auto increment integer as the issue ID instead of UUID, there are some tradeoffs involved. We will leave the discussion to a separate post.

## **Developer Experience**

Since Bytebase chooses open source, we do want to make it more accessible to the developers. So in the very beginning, we set a design goal for Bytebase:

> _Zero config, start with a single command within 10 seconds._

Bytebase achieves these by:

1. Use Golang as the backend, Golang is statically linked and requires no external dependency.
2. Use the embedding feature introduced in Go 1.16 to package the frontend files.
3. Use SQLite instead of the typical MySQL/PostgreSQL to store the data (this will be further discussed in a separate post).

The end result is the developer can just execute ./bytebase to start the entire Bytebase app. In fact, it usually takes **less than 1 second** from start to function.

### ASCII Banner

When startup process completes, the console will print out the Bytebase ASCII banner:

![_](/content/blog/announce-bytebase/banner-hello.webp)

First, this is for branding purpose. But more importantly, this is to allow user to easily identify the application status. Similarly, if Bytebase stops normally, it will print out the Bye ASCII banner:

![_](/content/blog/announce-bytebase/banner-bye.webp)

Both helps user to locate the Bytebase start/stop point among the abundant log entries.

### Let the product speak

Bytebase has built-in demo and read-only mode enabled by the respective --demo, --readonly flag. In fact, demo.bytebase.com is running with both --demo and --readonly. Bytebase is a tool built for database professionals with its own design philosophy. Even for those veterans, there is a learning curve. The demo mode helps the new user to skip the cold start phase, and can jump directly to play with the core feature set. As to the read-only mode, it was originally developed for demo.bytebase.com. We want to have a clean data to show each visitor the desired demo state, and to achieve this, we need to ensure the state is immutable, and won't be messed up by the previous visitor. Moreover, since the state is immutable, we can reference the specific demo.bytebase.com page from our doc site docs.bytebase.com as the supplement live example. Later on, Bytebase can also leverage the read-only mode to develop enterprise-grade features like maintenance window and deployment freeze.

## **Tool not Platform (yet)**

Right before the launch, we made a minor change during the last thorough code review

> _Tool not Platform (yet)_

![_](/content/blog/announce-bytebase/tool-not-platform.webp)

It has taken a lot of effort to bring Bytebase to the public. Yet, there is a lot of work ahead. We don't have any previous experience to run an open source community, neither do we have any marketing experience. The chosen tech stack and the data model still needs to stand the test of time. All in all, we still need to be pragmatic and focused on building a solid tool first.

## **Conclude**

It's already a quite long post and thanks for following us thus far. You can visit [https://github.com/bytebase/bytebase](https://github.com/bytebase/bytebase) to check out our project. If you happen to be a DBA or a developer looking for a database schema change management tool, give Bytebase a try and let us know what you think. Feedback appreciated.

Go back to build 🚀🚀🚀


The
Knowledge Base

Explore the standard for database development

Solutions & Platform

Comparisons

Legal

Resources

Company