The JET Blue Story
The Exchange Server product used the Extensible Storage Engine (ESE, aka JET Blue) to store data for decades. The story of the JET Blue (in contrast to JET Red which is used for Access database) can be read here (https://en.wikipedia.org/wiki/Microsoft_Jet_Database_Engine). In the acient days of data storage the ESE database was the best choice for storing mostly unstructured data with many dynamic properties.
The Messaging API (MAPI) had been developed in the 1990s to provide programmers with a set of unified interface for easier message exchange. The MAPI documentation at TechNet has been replaced by the current Outlook 2013 MAPI Reference. In todays world it is not easy to find reliable ressource about the original MAPI implementation. The only printed resource is Inside Mapi (Microsoft Programming Series) , ISBN 978-1572313125 , which has been published in 1996.
At Ignite 2015 Ross Smith VI joked about moving the Exchange storage engine to SQL. Back in the day with Exchange 2013 in production and Exchange 2016 coming, this was true. But Ross laid the tracks for the evolution of Exchange.
But it seems that the Exchange Product Team realized that in today's world with heavily standardized communication and less dynamic requirements than in the 1990s the days of JET blue are over. At the same time SQL Server evolved to mature database solution, capable of handling big data. The question was, if it can store SharePoint data, why not Exchange data. After twenty years of Exchange Server using the good ole ESE engine it was time to move on.
The SQL scripts that are used by Exchange to configure SQL are loacted in $exbin\SQL
CREATE TABLE [dbo].[MAPI_PROPERTIES]( [MAPI_PROPERTTY_ID] [int] IDENTITY(1,1) NOT NULL, [MAPI_PROPERTY_NAME] [nchar](127) NOT NULL, [IsWellKnownProperty] [bit] NOT NULL, [MAPI_TYPE_ID] [int] NOT NULL, CONSTRAINT [PK_MAPI_PROPERTIES] PRIMARY KEY CLUSTERED ( [MAPI_PROPERTTY_ID] ASC)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]GOALTER TABLE [dbo].[MAPI_PROPERTIES] WITH CHECK ADD CONSTRAINT [FK_MAPI_PROPERTIES_MAPI_TYPES] FOREIGN KEY([MAPI_TYPE_ID])REFERENCES [dbo].[MAPI_TYPES] ([MAPI_TYPE_ID])GOALTER TABLE [dbo].[MAPI_PROPERTIES] CHECK CONSTRAINT [FK_MAPI_PROPERTIES_MAPI_TYPES]GO
Exchange Server goes SQL
The current Exchange 2016 CU2 Preview supports an undocumented registry key to activate SQL Server support for Exchange. Personally I do not know, if this was supposed to be officially included in a public realease. So maybe the SQL support was made available by error and is already removed from the most current build again.
The famous SqueakyLobster registry key in has been used in Exchange 5.5 to troubleshoot performance issues. The new „Lobster“ key is used to activate hidden code in Exchange Server product. The name of the key is LobsterMapiDB.
This key activates support for Exchange modern storage. Without this key you won't be able to move mailboxes from ESE legacy storage to SQL modern storage.
It is assumed that a SQL Server 2014 instance is available. A SQL Server 2014 Express edition is sufficient for testing purposes.
Any changes to configurations or the registry should be validated in a test environment first. Never try this in production right away.
The high level steps required to activate SQL support for Exchange 2016 are:
- Create a configuration file to provide the SQL connection string
- Create a SQL server login for the Exchange Trusted Subsystem security group
- Add a registry key to the local Exchange Server registry
- Restart Information Store Service (MSExchangeIS)
- Execute PowerShell script to migrate mailboxes to SQL
The detailed steps are:
- Create a new config file named Microsoft.Exchange.Data.SQL.exe.config in $exinstall/bin
- Create a SQL login for Exchange Trusted Subsystem
CREATE LOGIN [DOMAIN\Exchange Trusted Subsystem] FROM WINDOWS
- Create a new DWORD named LobsterMapiDB in
and set the data value to 1
- Restart Exchange Information Store
- Use Move-MailboxToModernStorage.ps1 script to move selected mailboxes to modern storage
More can be found here:
- Migration scripts and config files are available at Github, https://github.com/Apoc70/LobsterMapiDB
- From crush to product documentation: The story of Squeaky Lobster, http://blogs.technet.com/b/exchange/archive/2006/06/15/3394229.aspx
- Setting „Squeaky Lobster“ performance counter settings, https://support.microsoft.com/en-us/kb/556030
- Extensible Storage Engine, https://en.wikipedia.org/wiki/Extensible_Storage_Engine
- Outlook 2013 MAPI Reference, https://msdn.microsoft.com/en-us/library/cc765775.aspx
- SQL Server 2014, https://www.microsoft.com/en/server-cloud/products/sql-server/
- Final information: https://goo.gl/QiTtDo
Enjoy Exchange for the next 20 years…