Skip to content

Instantly share code, notes, and snippets.

@kennwhite

kennwhite/mongodb-csfle-example-kmip-or-local-key.md

Forked from pkdone/mongodb-csfle-example-kmip-or-local-key.md
Created April 18, 2022 20:51
Show Gist options
  • Save kennwhite/fd17f8b523c8f32076548dcca5a82914 to your computer and use it in GitHub Desktop.
Save kennwhite/fd17f8b523c8f32076548dcca5a82914 to your computer and use it in GitHub Desktop.
Download ZIP
MongoDB CSFLE example using mongosh with a master key sourced from KMIP or a local file
Raw
mongodb-csfle-example-kmip-or-local-key.md

MongoDB Client-Side Field Level Encryption (CSFLE) Using KMIP or Local Master Key (with mongosh)

Assumptions

  • You have an accessible MongoDB deployment already running and accessible (self-managed or in Atlas)
  • You have the modern MongoDB Shell (mongosh) installed locally on your workstation
  • You have a KMIP Server running and accessible, if you don't intend to use a local keyfile (for an example of running and configuring a Hashicorp Vault development instance, see: Hashicorp Vault Configuration For MongoDB KMIP Use)

Configure Local Workstation Context Files

From a terminal, execute the code below after first:

  • Changing useLocalMasterKey to true if you don't want to use a KMIP managed master key and instead want to use a local keyfile
  • Changing url to match the cluster address of your MongoDB deployment
  • Changing kmipServer to match the address of your KMIP deployment (only necessary if not using a local keyfile)
# Clean out previously creayed files, if any
rm -f master_local_keyfile data*_key_id_file init_env.js people_schema.js

# Generate local master keyfile for CSFLE
echo $(head -c 96 /dev/urandom | base64 | tr -d '\n') > master_local_keyfile

# Initialise empty file which will later hold ids of data keys used for field encryption
touch data1_key_id_file data2_key_id_file

# Create a JS file to load constant values 
cat > init_env.js <<EOF
var useLocalMasterKey = false;
var url = "mongodb+srv://myuser:[email protected]/";
var localMasterkeyFile = "master_local_keyfile";
var kmipServer = "localhost:5696"
var dataKeysDBName = "csfle";
var dataKeysCollName = "dataKeys";
var dataKey1IdFile = "data1_key_id_file";
var dataKey2IdFile = "data2_key_id_file";
var dataKey1Name = "ssnEncKey"
var dataKey2Name = "contactEncKey"
var dataDBName = "personsdb";
var dataCollName = "people";
var localMasterKey = fs.readFileSync(localMasterkeyFile).toString().trim();
var dataKey1IdText = fs.readFileSync(dataKey1IdFile).toString().trim();
var dataKey2IdText = fs.readFileSync(dataKey2IdFile).toString().trim();
EOF

Define a JSON Schema to Represent a Person with Some Fields Marked as Encrypted

Execute from a terminal:

# Create a the JSON schema file for a person
cat > people_schema.js <<EOF
// Defines a JSON schema variable - IMPORTANT: variable dataKey1Id must already exist before loading this file
var peopleSchema = {
   "bsonType": "object",
   "properties": {
      "firstName": {"bsonType": "string"},
      "lastName":  {"bsonType": "string"},
      "ssn": {
         "encrypt": {
            "bsonType": "string",
            "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random",
            "keyId": [dataKey1Id]
         }
      },
      "address": {
         "bsonType": "object",
         "properties": {
            "street": {bsonType: "string"},
            "city": {bsonType: "string"},
            "state": {bsonType: "string"},
            "zip": {bsonType: "string"},
         }
      },
      "contact": {
         "bsonType": "object",
         "properties": {
            "email"   : {
               "encrypt": {
                  "bsonType": "string",
                  "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
                  "keyId": [dataKey2Id]
               }
            },
            "mobile"  : {
               "encrypt": {
                  "bsonType": "string",
                  "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
                  "keyId": [dataKey2Id]
               }
            }
         },
      },
   }
};
EOF

NOTE: The SSN field doesn't actually need to use randomized encryption because the cardinality is naturally high, meaning there's no conceivable common use case where a frequency attack can be used against it. It is used here just to show an alternative example, but in the real world, it would be desirable to use a deterministic encryption approach to improve queryability.

Generate CSFLE Data Keys in a MongoDB Vault Using External KMIP or Local Master Key

From a terminal, execute the code below after first:

  • Changing "ca.pem" to match the path of the Certificate Authority certificate file used by your KMIP server
  • Changing "client.pem" to match the path of the Private Key + Certificate file to be used to access the KMIP server
mongosh --nodb

// Load constants from file
load("init_env.js");

// Assemble metadata for connecting to MongoDB using CSFLE
var clientSideFieldLevelEncryptionOptions = {};

// Use either local master key file or KMIP managed master key
if (useLocalMasterKey) {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "local": {
        "key": BinData(0, localMasterKey)
      }
    }
  };
} else {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "kmip": {
        "endpoint": kmipServer      
      },    
    },
    tlsOptions: {
      kmip: {
        tlsCAFile: "ca.pem",
        tlsCertificateKeyFile: "client.pem",
      }
    }
  };
}


// Contect to MongoDB with CSFLE params
var connection = Mongo(url, clientSideFieldLevelEncryptionOptions);

// Create a data key vault and persiste 2 new generated data keys for encrypting fields differently
var keyVault = connection.getKeyVault();

if (useLocalMasterKey) {
    keyVault.createKey("local", {}, [dataKey1Name]);
    keyVault.createKey("local", {}, [dataKey2Name]);
} else {
    keyVault.createKey("kmip", {}, [dataKey1Name]);
    keyVault.createKey("kmip", {}, [dataKey2Name]);
}

// Get the internal ids to identify the 2 new data keys and save them to the local filesystem for future refernece
var dataKey1Id = keyVault.getKeyByAltName(dataKey1Name).toArray()[0]._id
var dataKey2Id = keyVault.getKeyByAltName(dataKey2Name).toArray()[0]._id
fs.writeFileSync(dataKey1IdFile, dataKey1Id.toUUID().toString());
fs.writeFileSync(dataKey2IdFile, dataKey2Id.toUUID().toString());

// Load the schema for the people collection and assocaite it with the colleciton to enforce 
// server-side validation which marks some fields to be encrypted by the generated key
load("people_schema.js");
var db = connection.getDB(dataDBName);
db.createCollection(dataCollName, {"validator": {"$jsonSchema": peopleSchema}});

exit();

Test Inserting Records with Encrypted Fields

From a terminal, execute the code below after first:

  • Changing "ca.pem" to match the path of the Certificate Authority certificate file used by your KMIP server
  • Changing "client.pem" to match the path of the Private Key + Certificate file to be used to access the KMIP server
mongosh --nodb

// Load constants from file, data key id from file and people schema from file
load("init_env.js");
var dataKey1Id = UUID(dataKey1IdText);
var dataKey2Id = UUID(dataKey2IdText);
load("people_schema.js");

// Get the people schema into an object that associates it with a specific database collection
var schemaMap = {};
schemaMap[dataDBName + "." + dataCollName] = peopleSchema;

// Assemble metadata for connecting to MongoDB using CSFLE, including client-side schema definition
var clientSideFieldLevelEncryptionOptions = {};

// Use either local master key file or KMIP managed master key
if (useLocalMasterKey) {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "local": {
        "key": BinData(0, localMasterKey)
      }
    },
    "schemaMap": schemaMap        
  };
} else {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "kmip": {
        "endpoint": kmipServer      
      },    
    },
    tlsOptions: {
      kmip: {
        tlsCAFile: "ca.pem",
        tlsCertificateKeyFile: "client.pem",
      }
    },
    "schemaMap": schemaMap        
  };
}


// Contect to MongoDB with CSFLE params and get handle onto data collection
var connection = Mongo(url, clientSideFieldLevelEncryptionOptions);
var db = connection.getDB(dataDBName);

// Insert person records
db[dataCollName].insertOne({
   firstName: 'Alan',
   lastName:  'Turing',
   ssn:       '901-01-0001',
   address: {
      street: '123 Main',
      city: 'Omaha',
      zip: '90210'
   },
   contact: {
      mobile: '202-555-1212',
      email:  '[email protected]'
   }
});

// Show current persons records
db[dataCollName].find();

exit();

In the output above you should see the real unencrypted values of the fields ssn, mobile and email.

Use a Different MongoDB Shell with no CSFLE Awareness to View the Persisted Data

From a terminal, execute the code below after first changing the shown MongoDB URL to match your MongoDB deployment:

mongosh "mongodb+srv://main_user:[email protected]/"
use personsdb;
db.people.find();

In the output above you should see the encrypted values for the fields ssn, mobile and email.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment