NetApp NSE Setup and Management

NOTE: NSE must be ALL or NONE NSE per HA-Pair

Authentication Keys (AK) and changes to them do not affect the disk encryption keys
When a system is first brought up, the NSE disks are openly available to the system without need for authentication. The disks themselves automatically encrypt data written to them and decrypt it when read and maintain these disk encryption keys (AKA media encryption keys) within themselves. The controls are not yet set to protect a disk that leaves the system. The system may be operated in this unprotected mode indefinitely. The NSE disks simply act like other disks.

When the servers are made available and the required SSL/TLS certificates are properly installed, the setup of the connections between the KMIP servers and the cluster is made. Thereafter, authentication keys can be created and the controls in the disks set to protect the data. Then, if the disks are power-cycled, such as would happen if a disk is removed and placed on another system, that system cannot give the required AK (safely on an SSL-protected key server) to unlock access to the data.

Modifying authentication keys does not affect the encryption keys. Data that is written to the disks in the period before KMIP server setup and AK changes is still present. Once the controls are set, then all data on the disks is protected, whether it existed before or after the protections were applied.

The disks come with a default key, called the Manufacture Secure ID (MSID), that is unique to each disk. It is electronically readable from the disk, so it provides no protection on its own. This might be what the questioner referred to as “the open key.” When Data ONTAP modifies the AK to a new value the MSID can no longer be used to access the disks, if it should leave the system.

Storage encryption is at the disk firmware on self-encrypting disks (SEDs). SEDs run in unprotected or protected mode (encrypted). Protected mode requires key manager authentication after power-on. There is no noticeable performance decrease or boot time increase. Furthermore, all Data ONTAP storage efficiencies (i.e. dedupe, compression) are not affected.

You can specify up to 4 key servers during or after setup. You should have at least 2 key servers. If you have production and DR site the key managers are clustered together; this is a common setup.

SEDs have two additional features in addition to encryption

1. Sanitize (for return) changes the encryption key to a new unknown key
2. <strongDestroy changes the encryption key to a random unknown key and permanently locks the disk

• Gemalto SafeNet Professional Services are REQUIRED for installation of the key manager
• Certifications and reasons for NSE
  • FIPS 140-2 level 2
  • PCI
  • HIPAA
  • FISMA
  • CA SB1386
• cDOT 8.2.1+ (7-Mode 8.1.1+)
  • 8.2.1 disk encrypt commands are run at node shell
  • 8.3+ disk encrypt commands MUST be run at cluster shell
• When a drive is replace by RMA, make sure to run the storage encryption disk modify command
• FIPS compliance mode 8.3.1+ ONLY
  • FIPS key can be different from the drive authentication key
  • Sanitizing or resetting to factory settings removes FIPS compliance mode
  • If set to FIPS compliance and downgrade <8.3.1, disable FIPS on all SEDs storage encryption disk modify -disk * -fips-key-id 0x0
• KMIP 1.0 and 1.1 supported
• 256-bit AES encryption depending on drive
• No MetroCluster support
• Do NOT use 10Gb interfaces for KMIP
  • e0M is recommended because different models might not have other interfaces available during the boot process
• Cluster Integration
  • 1 authentication key per node
  • Each disk has its own DEK (Drive Encryption Key)
• Up to 128 Authentication keys per Cluster with a warning threshold of 100NOTE: See the Interoperability Matrix for supported key management servers. External Key Management is REQUIRED with NSE for Data ONTAP <9.0. Data ONTAP 9.0 has onboard key managementHow to determine whether you need an external key management serverYou should set up an external key management server if any of the following are true for your environment:
  • Your encryption key management solution must comply with Federal Information Processing Standards (FIPS) 140-2.
  • You need a multi-cluster solution.  With an external key management server, you can have a single key server solution that supports multiple clusters with centralized management of all encryption keys.
  • You need support of industry OASIS Key Management Interoperability Protocol (KMIP) standard.
  • Your business requires the added security of storing your encryption keys on a system or in a location different from the data.  With onboard key management, the data and the encryption keys are stored on the same system.  With an external key server, the data and keys are stored separately.

  Current Support (subject to change)

  • Gemalto SafeNet KeySecure k460
    • 30K list
    • Physical
    • FIPS-140-2 level 3
    • 1 million keys
    • LDAP, AD, Syslog forward
  • Gemalto SafeNet KeySecure k150v
    • 10K list
    • Virtual
    • FIPS-140-2 level 1
    • 25,000 keys (100 per NetApp cluster)
    • LDAP, AD, Syslog forward
  • BM TKLM/SKLM (not sold by NetApp)
  • Vormetric DSM (PVR ONLY and not sold by NetApp)

Setting Up NSE

SSL

• Authentication cluster to KMIP is over SSL and we need both public and private SSL certificates for the cluster and the public SSL certificate of the KMIP server.
  • Synchronize the time between all devices including the server creating the certs
  • SSL client cert must not be password protected
  • Common name of certs can not contain “_” underscore
    

    Cert	Filename	Notes
    HA-Pair Public	client.pem
    HA-Pair Private	client_private.pem	can be a passphrase
    KMIP Public	keymanagementserver_ip.CA.pem

Install Certificates on the cluster

SSL client certificate for the cluster

• Prompts for the public and private key

Install public certificate for the root certificate authority (CA) on the SSL server (external KMIP server)

• Enter IP by subnet if multiple, for example 192.168.150.10, 192.168.150.11, 192.168.150.12 use “192.168.150.0” or use “0.0.0.0” as a catch all
• If different root CAs, repeat the command for each KMIP public certificate SSL certificates expire.  Remove old certs before installing new certs. 
  Remove old certs

  ::> security key-manager delete -address x.x.x.x # from key management server
::> security certificate delete -type client -vserver svm_name -common-name fqdn_or_custom_common_name -ca certificate_authority -subtype kimp-cert # from cluster's client certs
::> security certificate delete -type server-ca -vserver svm_name -common-name fqdn_or_customer_common_name -ca certificate_authority -subtype kmip-cert # remove all installed key management server certs - enter for EACH key management server

Go back to top of this sub-section and run security certificate install above for -type client and type server-ca, and then repeat security key-manager add below for additional key manager servers
Add a key manager server (best practice = 2 total; max = 4)

Delete management server

Verify key management server links

Setup Encryption with the key management setup wizard (SSL certificates MUST be installed). Also run this when changes are made after initial setup

Prompts	Default
TCP port	5696
Interface (1Gb)	e0M
IPv4	yes
IP	x.x.x.x
Mask	x.x.x.x
GW	x.x.x.x
IPv6	no

Disk Encryption (create-key and modify disk to key)

Create Key with -key-tag (=clustername by default)

• Manual

  ::> security key-manager create-key -prompt-for-key true -key_tag key_tag

• Automatic

  ::> security key-manager create-key -key_tag key_tag

Assign Key to Disks (and also after an RMA of a broken disk)

Turn ON FIPS (OPTIONAL) after disk encryption above

Turn OFF FIPS, Move Volumes, Reset drives)

• vol move data
• Sanitize disks

• Reset to factory original settings (if disks have PSID)

Retrieve Authentication Keys (new node or node that was halted)

When you create a new key, ONTAP automatically restores the key so the new key is available on all nodes. This is for NEW or HALTED nodes that do not have the key.

Return SEDs to unprotected mode (default MSID keys)

Disable FIPs

Change auth key to default MSID

Remove key manager servers

View and delete SSL Certs

• Repeat for each KMIP server

Return to service a disk with no auth key (can’t restore data, but can re-use drive)

Two Methods

1. NO FIPS, or FIPS key is still available

STOP if FIPS compliance mode and FIPS key is available. Continue if not FIPS

2. SEDS are in FIPS compliance mode and keys are not available and IF the disk PSID is printed on the disk label

Destroy / Sanitize (Shred) Before return RMA (works on spare and broken disks unless you use -force-all-states true)

• to run against ALL drives in the cluster (be careful)

Destroy Disk – EOL (spare and broken disks ONLY)

• DESTROY the PSID on the DISK LABEL and all photos/copies of the drive label

EMERGENCY SHREDDING – Destroy Disk (even if no power and no key-mgmt)

Power is Available and you have time

Power is Available and you DO NOT have time

• NODES PANIC

No Power

Destroy Key Manager Keys online , or if no power on key manager destroy the smart card in the key management server

Boot a system with the key manager not available – save the passphrase from original setup

If a system is halted and moved to a new network, change the security key-manager setup before halt, or use the loader prompt to set the passphrase, then rerun security key-manager setup. The values for the passphrase and keyid variables are documented during the Key-Manager setup process. Once these are defined, Data ONTAP will boot. Change the Custer and Node IPs and re-run the key-manager setup.

OLD METHOD prior to 8.3.x. “security key-manager setup” wizard…. no boot loader needed with the cluster command method

https://kb.netapp.com/support/index?page=content&id=1012619&actp=search&viewlocale=en_US&searchid=1348162615003

strong>How to set pre-boot environment variables for NetApp Storage Encryption using KMIP servers

KB ID: 1012619 Version: 14.0 Published date: 02/08/2016

Description

Setting pre-boot environmental variables are necessary for NetApp Storage Encryption (NSE) storage systems. The NSE bootarg bootarg.storageencryption.support true is set at the factory if NSE drives are present in the controller. The other bootargs listed below are automatically created when the key_manager setup (nodeshell \ 7-Mode) or the security key-manager setup (clustered Data ONTAP 8.3.1 and later) command is run.

If the storage system is unable to correctly contact the KMIP Key Management Server, it will not be able to boot. This is because the required boot information cannot be decrypted and will therefore be unreadable.

Procedure

1. Log in to the storage system using the service processor (SP, RLM or BMC) or through the serial console port. This will be required for the completion of all additional steps to correctly add/verify the pre-boot settings for the KMIP server.
2. When prompted, press Ctrl+C to halt the boot process. This will bring up the loader prompt. If the loader prompt does not appear, the storage system will have to be rebooted. Ctrl+C should be used at the first prompt to halt storage system boot up, and not at the second prompt.
3. Access the pre-boot environment variables from the loader prompt. The command printenv will show all the existing pre-boot variables.

   Loader_A> printenvUse this command to list all pre-boot variables, and look for the following five:

   kmip.init.interface xxx
   kmip.init.ipaddr xxx.xxx.xxx.xxx
   kmip.init.netmask xxx.xxx.xxx.xxx
   kmip.init.gateway xxx.xxx.xxx.xxx
   bootarg.storageencryption.support true

   This string is required for NSE at all times. Do not remove this string or set it to false. If the string is missing, Data ONTAP will discover the NSE hard drives as failed devices.

4. Use the setenv command to set the variables for the pre-boot environment:

   Loader_A> setenv kmip.init.interface e0M
   (All values are case-sensitive. For example, in e0M, the M must be in upper case)
   
Loader_A> setenv kmip.init.ipaddr xxx.xxx.xxx.xxx
Loader_A> setenv kmip.init.netmask xxx.xxx.xxx.xxx
Loader_A> setenv kmip.init.gateway xxx.xxx.xxx.xxx
Loader_A> setenv bootarg.storageencryption.support true


   Important Notes:

   1. The four kmip.init.* values must all be set.  If any one of the values is not entered, the whole interface will be treated as unconfigured.  The interface IP used is required to be a dedicated IP address that will not be used when Data ONTAP is loaded; this is to prevent having duplicated IPs during Controller Takeover.
   2. Beginning with Data ONTAP 8.1, NSE supports FAS2040, FAS2240, FAS32xx, FAS62xx, V32xx, and V62xx.  The pre-boot environment variables applies to all NSE FAS Filers.  Depending on the FAS platform, the FAS may or may not have an e0M interface.  e0M is the recommended interface to use if the FAS platform has the management NIC.
   3. Save the changes to the pre-boot environment.  Otherwise, they will only be valid for a single boot of the filer.  Use the command saveenv to save changes to the pre-boot variables.

      Loader_A> saveenv

5. Boot the storage system from the loader prompt. Use the following command to resume the storage system boot up:

   Loader_A> boot_ontap or bye