Data desensitization

Background information

Data desensitization changes the forms of sensitive information, such as names, ID card numbers, mobile phone numbers, landline numbers, bank accounts, and email addresses. You can set data desensitization rules to protect sensitive data.

Concepts

• Data desensitization: the process of deidentifying, masking, or replacing sensitive data by using special algorithms and techniques during data processing and storage to protect data security and prevent data breach.

• Dynamic desensitization: the process of desensitizing sensitive data in real time. The data is desensitized only for a database query. The source data in the database is not modified. Dynamic desensitization is commonly used in production environments to maintain the integrity and accuracy of raw data while avoiding the risk of data breach. The drawback of dynamic desensitization is its limited processing speed, which may affect the database query efficiency.

• Static desensitization: the process of preprocessing sensitive data and storing the processed data in storage media, such as a database. Static desensitization is usually used in testing, development, and demonstration environments. It protects sensitive data from being viewed by unauthorized personnel and avoids the legal liability of data breach. The benefit of static desensitization is the fast processing speed and high query efficiency. However, the raw data is overwritten, so the data accuracy is undermined.

• Desensitization algorithm: an algorithm used to desensitize data. A desensitization algorithm can effectively protect the security of sensitive data, avoid information breaches, and retain data formats and structures for data query and usage.

• Identification rule: a rule used to automatically identify sensitive data for dynamic desensitization. ODC automatically scans data and identifies sensitive columns based on the added identification rules.

• Sensitive column: a column that contains sensitive data in a database table.

Execution process

1. A project administrator views the built-in desensitization algorithm and tests the desensitization effect in Desensitization Algorithm under Security Specifications.

2. The project administrator manually adds sensitive columns on the Sensitive Data tab for a project. To add sensitive columns by using the automatic scanning feature, you must first create an identification rule.

3. A user views the table data, queries data in the SQL window, exports a result set, exports a ticket, and submits a database change ticket. The sensitive columns in the data are desensitized.

Prerequisites

• The project administrator and DBA can manage sensitive columns and identification rules.

• All users can view and test the desensitization effect. Users cannot create, edit, or delete a desensitization algorithm.

Manage sensitive columns

Add sensitive columns

Assume that you want to desensitize the email and mobile_phone columns in the student table in the odc_test database.

1. In the project collaboration window, choose Project > Sensitive Data > Sensitive Columns > Add Sensitive Column > Add Manually.

2. Add sensitive columns by taking one of the following means, and then click Submit.

   • Method 1: Manually add sensitive columns.

   • Method 2: Scan for sensitive columns automatically.

     Note

     Before automatically scanning for sensitive columns, you must create an identification rule. For more information, see Manage identification rules.

3. In the Sensitive Data list, you can view and enable the added sensitive columns.

Edit sensitive columns

As shown in the preceding figure, in the Sensitive Data list, click Edit in the Actions column to modify the desensitization algorithm.

Delete sensitive columns

On the Sensitive Data tab, click Delete in the Actions column.

Manage identification rules

Identification rules facilitate the management of sensitive data. After you manually add sensitive columns, you can add custom identification rules to automatically scan for sensitive columns. An identification rule is a matching condition that ODC uses to identify sensitive columns. ODC allows you to define an identification rule in three aspects: path, regular expression, and script.

• Path: Use the database name, table name, or column name as the identification object. You can enter a path identification expression as an identification rule. Use periods (.) to separate database and table column names. Use asterisks (*) as wildcards. Use commas (,) to specify multiple rules.

  Parameter	Required?	Description
  Rule Name	Yes	The name of the rule. The name can contain up to 64 characters.
  Rule Status	Yes	The status of the rule. Valid values: Enabled and Disabled.
  Matching Rule	Yes	The rule that qualifies a data column as a sensitive column. For example, the rule *.*.mobile_phone qualifies any column named mobile_phone in all databases and tables.
  Exclusion Rule	No	The rule that disqualifies a data column as a sensitive column. Notice ODC checks a data column against the exclusion rule first, and then the matching rule, to determine whether it is a sensitive column.
  Desensitization Algorithm	Yes	The default desensitization algorithm for masking sensitive columns.
  Description	No	The description of the identification rule.

• Regular expression: Use the database name, table name, column name, or column comments as the identification object. You can enter a path identification expression as an identification rule. You can enter a regular expression as an identification rule.

  Parameter	Required?	Description
  Rule Name	Yes	The name of the rule. The name can contain up to 64 characters.
  Rule Status	Yes	The status of the rule. Valid values: Enabled and Disabled.
  Identification object - Database name	No	Enter a regular expression for matching a database name. For example: * represents a database of any name.
  Identification object - Table name	No	Enter a regular expression to match a table name. For example, e[a-z]?.* represents any table with a lowercase English name that starts with the letter e.
  Identification object - Column name	No	Enter a regular expression to match a column name.
  Identification object - Column comments	No	Enter a regular expression for matching column comments.
  Desensitization Algorithm	Yes	The default desensitization algorithm for masking sensitive columns.
  Description	No	The description of the identification rule.

• Script: Use the database name, table name, column name, column comments, or data type as the identification object. You can enter a Groovy script as the identification rule.

  Notice

  The output of the script must be a Boolean value, either True or False.

  Parameter	Required?	Description
  Rule Name	Yes	The name of the rule. The name can contain up to 64 characters.
  Rule Status	Yes	The status of the rule. Valid values: Enabled and Disabled.
  Groovy Script	Yes	A script that determines whether a column is a sensitive column. The script must comply with the Groovy syntax standard.
  Desensitization Algorithm	No	The default desensitization algorithm for masking sensitive columns.
  Description	No	The description of the identification rule.

  ODC has a built-in object named column for users to reference in Groovy scripts. The following table describes the attributes of the object.

  Attribute	Type	Description
  schema	String	The name of the database to which the column belongs.
  table	String	The name of the table to which the column belongs.
  name	String	The name of the column.
  comment	String	The comment of the column.
  type	String	The data type of the column.

Here are some sample scripts of general identification rules:

• Addresses

  if (("varchar".equals(column.type) || "char".equals(column.type))) {
      if (column.name.indexOf("address") >= 0) {
          return true;
      }
      if (column.comment != null &&
              (column.comment.toLowerCase().indexOf("address") >= 0
                      || column.comment.indexOf("address") >= 0
                      || column.comment.indexOf("home address") >= 0
                      || column.comment.indexOf("location") >= 0)) {
          return true;
      }
  }
  return false;
  

• Mobile phone numbers

  if (column.name.length() == 11 && ("varchar".equals(column.type) || "char".equals(column.type))) {
      if (column.name.indexOf("phone") >= 0 || column.name.indexOf("mobile") >= 0) {
          return true;
      }
      if (column.comment != null &&
              (column.comment.toLowerCase().indexOf("phone") >= 0
                      | | column.comment.indexOf("phone") >= 0
                      || column.comment.indexOf("mobile") >= 0
                      || column.comment.indexOf("mobile") >= 0)) {
          return true;
      }
  }
  return false;
  

• ID card numbers

  if (column.name.length() >= 15 && ("varchar".equals(column.type) || "char".equals(column.type))) {
      if (column.name.indexOf("id_number") >= 0 || column.name.indexOf("identity_card") >= 0) {
          return true;
      }
      if (column.comment != null &&
              (column.comment.toLowerCase().indexOf("identity card") >= 0
                      || column.comment.indexOf("ID card") >= 0)) {
          return true;
      }
  }
  return false;
  

Add an identification rule

Assume that you want to add an identification rule for the mobile_phone column in the student table in the odc_test database as the administrator.

1. In the project collaboration window, choose Project > Sensitive Data > Identification Rules > Create Identification Rule.

2. In the Create Identification Rule dialog box, specify the rule name, rule status, and identification method.

   For example, odc_test*.student.*a,*.*.mobile_phone matches the mobile_phone column in the student table in the odc_test database.

3. In the Identification Rules list, you can view and enable the created identification rule.

Manage identification rules

As shown in the preceding figure, you can click Edit or Delete to modify or delete an identification rule.

View desensitization algorithms

On the project collaboration page, choose Security Specifications > Desensitization Algorithms to view the desensitization algorithms supported by ODC.

The following table describes the desensitization algorithms supported by ODC.

Scenarios

The added sensitive columns take effect in data export, database change, and SQL window execution scenarios.

Scenario 1: Data export

Assume that you want to export the student table from the dc_test database and check the desensitization result.

1. In the left-side navigation pane of the SQL development window, submit a ticket to export the student table. In the export task list, click View.

2. In the lower-right corner of the export task details page, click Download.

3. View the student table on your local disk.

Scenario 2: Database changes

Assume that you want to insert data into the student table and check the desensitization result. Data desensitization is automatically enabled in this case.

1. In the left-side navigation pane of the SQL development window, submit a ticket to create a database change task that inserts data into the student table.

2. In the left-side navigation pane of the SQL development window, check the student table in the odc_test database under . The data is desensitized.

Scenario 3: SQL statement execution in the SQL window

Assume that you want to insert data into the student table and check the desensitization result. Data desensitization is automatically enabled in this case.

1. In the SQL window, edit an SQL statement to insert data into the student table.

2. On the Results tab, view the data in the student table. The data is desensitized.

References

• Export data

• Edit and execute SQL statements

• Manage database changes