Secrets

Storing and using secrets through Mage

In various surfaces in Mage, you may be asked to input config for certain integrations such as cloud databases or services. In these cases, you may need to input a password or an api key, but you don’t want it to be shown in plain text. To get around this issue, we created a way to store your secrets in the Mage database.

Your secrets are encrypted before being stored into the Mage database. The encryption key will be stored in the Mage data folder which by default will be created at ~/.mage_data. If you need more secure encryption, we recommend using a secrets manager.

Secret management

Creating secrets

To manage your secrets, you’ll want to go to the edit page for a pipeline. In the sidekick, you should see a Secrets tab. To create a secret, press the New button on this tab. Input a name for the secret and the value of the secret, and press Enter or Return to save.

The secrets can be shared across the project that they are created in.

Pipeline level secrets coming soon…

Secrets in different environments

Secrets are stored to the Mage database. If you are using the same database across multiple Mage environments, i.e. development and production, the secrets will not be shared across the environments unless you shared the same Mage data directory for every environment.

Each secret will be saved to the database with a name and a uuid. The name will be used to identify the secret, and the uuid is used to identify the environment.

Example:

Development environment
project
└───mage_data
│   └───secrets
│   │   │   key: abc123
│   │   │   uuid: dev-uuid

Production environment
project
└───mage_data
│   └───secrets
│   │   │   key: def456
│   │   │   uuid: prod-uuid

Secrets table in database

Name	Uuid	Value
secret1	dev-uuid	sooper dooper secret string
secret2	dev-uuid	xXxSecreTxXx
secret3	dev-uuid	dev only secret
secret1	prod-uuid	real production secret
secret2	prod-uuid	m4g3r0x!
secret1	random	random secret

Command	Development Value	Production Value
get_secret_value('secret1')	sooper dooper secret string	real production secret
get_secret_value('secret2')	xXxSecreTxXx	m4g3r0x!
get_secret_value('secret3')	dev only secret	None

Coming soon:

• secrets scoped to a pipeline

Using Secrets

You can use the following syntax to have Mage interpolate the secret when reading from the config:

field_name: "{{ mage_secret_var('your_secret_name') }}"

You can also fetch the secret value in a Mage code block by importing a helper method:

from mage_ai.data_preparation.shared.secrets import get_secret_value

get_secret_value('your_secret_name')