Understanding Machine Key Rotation using the new automatic SharePoint timerjob

- - 22 Comments

With the September 2025 CU, SharePoint introduced an automatic machine key rotation timer job to enhance security by periodically updating the ASP.NET machine keys used for view state encryption and validation. This post explains how the job works, how keys are updated in memory, and how you can monitor and troubleshoot the process.

How the Automatic Machine Key Rotation Job works

The automatic machine key rotation job is designed to improve security while minimizing downtime and performance impact. Here’s the detailed process:

  1. Timer Job Execution
    • Runs weekly by default (configurable via Central Administration)
    • Generates a new set of machine keys (validationKey and decryptionKey) for the entire farm
  2. Centralized Storage in Configuration Database
    • Instead of writing keys to each web.config, the job updates the configuration database.
    • This ensures centralized management and avoids triggering application domain restarts.
  3. Propagation to Configuration Cache
    • Each SharePoint server retrieves the updated keys from the configuration database and stores them in its local configuration cache.
  4. In-Memory Update in Worker Processes
    • The IIS worker processes (w3wp.exe) monitor the configuration cache.
    • When new keys are detected, they update their in-memory configuration without requiring an IIS reset or app pool recycle.

Why Don’t We See Changes in web.config?

This is by design. The Machine Key Rotation Job does not modify the web.config file. The keys are updated in memory only, which prevents performance problems through application domain recycles.
If you check the last modified date of the web.config file after the job runs, it will remain unchanged.

What Happens After Encryption Keys Are Updated?

Once the new keys are active in memory, all new requests use them for view state encryption and validation.

If a user had an open page before the rotation, the browser may still hold view state encrypted with the old key. Submitting that page can cause invalid view state errors.
Solution: Refresh the page to load a new view state encrypted with the current keys.

Manual Rotation vs. Automatic Rotation

  • Automatic Job: Updates keys in memory only. No web.config changes.
  • Manual Rotation:
    • Use Set-SPMachineKey to generate new keys and update web.config files.
    • If required use Update-SPMachineKey to replicate existing keys across the web.config files on all servers in the farm (does not generate new keys).

Troubleshooting

Using PowerShell

To verify the current machine keys for all web applications in the configuration cache and in the web.config, use the following PowerShells script I created:

Important:

  • After a machine key rotation was done using the Machine Key Rotation job it is expected that the machine keys emitted by both scripts are different, as the timer job DOES NOT update the web.config files.
  • After a machine key rotation was done using the Set-SPMachineKey PowerShell CmdLet it is expected that the machine keys emitted by both scripts are identical for the web application where the new machine keys were generated as the CmdLet DOES update the web.config file.
  • Update-SPMachineKey is irrelevant after the Machine Key Rotation job has been executed at least once in the farm as it will update the web.config machine keys which are no longer being used.

The Machine Key Rotation Job Timer job can found in the Central Administration Website and can also be displayed using PowerShell:

get-sptimerjob | ? {$_.Name -like "job-config-machine-key-rotation"} | fl

The Configuration Refresh Timer job which replicates the content to all servers in the farm is a hidden system timer job which is NOT visible in the Central Administration website – but you can find it using PowerShell:

get-sptimerjob | ? {$_.Name -like "job-config-refresh"} | fl

ULS Log Monitoring

When machine keys are rotated using the Machine Key Rotation job, SharePoint logs specific events in the ULS logs.
Below is the sequence of events you should find in the ULS log for a successful machine key rotation.

Information:
To troubleshoot machine key rotation issues, it’s essential to collect a merged log from all servers. Since different operations occur on different servers, a combined log allows you to view the entire sequence in a single file.A merged log can be generated using the following PowerShell command:

merge-splogfile -path "C:\temp\FarmMergedLog.log" -StartTime "MM/DD/YYYY HH:MM" -EndTime "MM/DD/YYYY HH:MM"

See here for more possible options for this command:

To analyze and filter ULS logs use UlsViewer

Information:
The GUID following MachineKeyConfiguration_ in some of the messages below represents the ID of the web application for which a new Machine Key was generated.
You can retrieve the Web Application ID by running the following PowerShell command:

(get-spwebapplication http://url-to-web-app).Id
Machine Key Rotation Timer Job

This timer job runs once a week by default. It generates new machine keys (for encryption and validation) and stores them in the configuration database.
The job executes on a single server in the farm, which must have the AllowServiceJobs property set to True for its TimerService instance.
You can check this by running the following PowerShell command:

(get-spfarm).TimerService.Instances | select Server,AllowServiceJobs
  • Indication that the Machine Key Rotation Job was started 
    Event ID: xmnv
Level: Medium
Process: OWSTIMER.EXE
Category: Logging Correlation Data
Message: Name=Timer Job job-config-machine-key-rotation
  • Indication for successful generation of new machine keys for a web application 
    Event ID: 4l4ri
Level: High
Process: OWSTIMER.EXE
Category: Runtime
Message: MachineKeyConfiguration_10110dbf-8776-4721-b4b2-abf8895bb5bd password changed. Needs update.
  • Indication for successfull update of machine keys for a web application in the configuration database 
    Event ID: 4l4rh
Level: High
Process: OWSTIMER.EXE
Category: Runtime
Message: Successfully persisted MachineKeyConfiguration_10110dbf-8776-4721-b4b2-abf8895bb5bd credential.
Configuration Refresh Timer Job

This timer job copies configuration data from the configuration database to the local configuration cache on each server in the farm.
It runs every 15 seconds on ALL servers in the farm.

If this timerjob does NOT run on all machines in the farm it might be that the TimerService instance is disabled on some servers in the farm.

You can check if all TimerService Instances are only by running the following PowerShell command:

(get-spfarm).TimerService.Instances

If you find a server where the TimerService instance is not Online you need to set it Online using this command:

(get-spfarm).TimerService.Instances |? {$_.Status -ne "Online"} |% {$_.Status = "Online"; $_.Update()}
  • Indication that the Configuration Refresh Timer Job was started 
    Event ID: bjgst
Level: Medium
Process: OWSTIMER.EXE
Category: Config Cache
Message: SPConfigurationRefreshJobDefinition.Execute: Refreshing timer config cache with refreshCacheFlags=[Default, SkipSetFlagOnFailure], newestVersion=[15186],
  • Indication that the machine keys for a web application have been replicated to the local filesystem configuration cache 
    Event ID: a3c26
Level: Medium
Process: OWSTIMER.EXE
Category: Config Cache
Message: Entering Monitored Scope (PutObjectInFileSystem: Storing object [SPSecureDBCredential Name=MachineKeyConfiguration_10110dbf-8776-4721-b4b2-abf8895bb5bd] with id ...
Machine Key update in the IIS worker process.

At the start of each HTTP request, the system checks whether the machine keys for the current web application have changed in the configuration cache. If changes are detected, the keys are updated in memory within the IIS worker process (w3wp.exe).

  • Indication that Machine keys in the worker process were replaced with the new machine keys from the configuration cache 
    Event ID: 7cvqt
Level: Medium
Process: W3WP.EXE
Category: Runtime
Message: Successfully updated machine key.

If any of these events is missing, investigate further by reviewing potential error messages.

22 Comments


  1. A nice article. Thank you!

    Please fix:
    – web application Id example is missing an open bracket;
    – “Indication that the Configuration Refresh Timer Job was started” appears twice.

    Reply

    1. Thanks Atis!
      Fixed both errors. 🙂

      Reply

  2. This is awesome. Thank you so much for putting everything “Machine keys” at one place.

    Reply

  3. Hi Stefan,

    We jumped over the september 2025 CUs because of all the problems reported. We started to update our first server with the october 2025 CUs. It went well. This server is a sole one that has all the roles in his farm (we’ll update our larger farms eventually).

    I searched the events that you described to see if all was ok with machine key rotation on this server.

    I can find the events about (so that’s fine) :
    Indication that the Machine Key Rotation Job was started
    Indication for successful generation of new machine keys for a web application (4 times, we have 4 web apps)
    Indication for successfull update of machine keys for a web application in the configuration database (4 times, we have 4 web apps)

    I cannot find these ones :
    Indication that the Configuration Refresh Timer Job was started
    Indication that Machine keys in the worker process were replaced with the new machine keys from the configuration cache

    I think that it’s probably because i have only one server in the farm ? Could you confirm this ?

    Thanks !

    Jeff

    Reply

      1. Thanks Stefan for the quick reply.

        I tested the two scripts on 6 servers (standalone farms, one server in each farm) and they work well on 4 of them but for two, the script DumpWebConfigMachineKeys.ps1 works fine, but the script DumpConfigDBMachineKeys.ps1 (config cache) returns the error :
        Object reference not set to an instance of an object.

        For the script DumpWebConfigMachineKeys.ps1, i had to adjust the web root paths (because we don’t use the default C:\Inetpub) for it to work but it does.

        For the script DumpConfigDBMachineKeys.ps1, would there be a way to troubleshoot ?

        Thanks again !

        Jeff

        Reply

        1. Hi Jeff,
          interesting – sounds to me as if on these machines the machine keys have not arrived.
          This is most likely the line:

          string bothKeys = cred.Password;

          if cred is null you will get the above error.

          I have updated the script. Please download it from scratch and run it in a new PowerShell windows.
          Should now tell you if my assumption is correct.

          Cheers,
          Stefan

          Reply

          1. I downloaded and ran the new version of the script and you are correct, it returns for each web app :
            No machine keys found for web application in config cache

            you assumption is spot on.

            Jeff


          2. Hi Jeff,
            this means that the config-refresh timerjob does not run on this machine.
            This can happen if the TimerService Instance on this machine is Offline.
            Please check using this command:
            (get-spfarm).TimerService.Instances

            I have updated the article above to includes this check and also the command to fix it if required.
            Cheers,
            Stefan


  4. Hi Stefan,

    On both servers where i get the error :
    No machine keys found for web application in config cache

    If i check for the TimerService instance, it is online (on both servers) :
    PS D:\Applications\Scripts> (get-spfarm).TimerService.Instances

    TypeName Status Id
    ——– —— —
    Microsoft SharePoint Foundati… Online 52871029-52c1-4447-a40c-7a014e83c919

    Jeff

    Reply

    1. Hi Jeff,
      please double check service.msc that the SharePoint Timer Service is running.
      If yes, I would recommend to open a support case with Microsoft to get this investigated.
      Cheers,
      Stefan

      Reply

  5. The SharePoint Timer Service is ok on both servers.

    On one of the two servers, i manually executed the timer job and afterward, the DumpConfigDBMachineKeys.ps1 script shows me the keys for each web app.

    For the other server, since we updated it yesterday, i’ll wait till next wednesday to check if the timer Job was run and if not, i’ll open a case with Microsoft as you suggested.

    Jeff

    Reply

    1. Hi Jeff,
      the machine key rotation job will always only run on one server in the farm. And not necessarily the server where you started it from. It is the config refresh job which replicates the details from the config db to the local machines config cache. The script check only the config cache on the file system – not the info in the config db itself. The config refresh job runs every 15 seconds. So if it is not there now, I do not see how it will be there next Wednesday.
      Cheers,
      Stefan

      Reply

      1. My 6 servers each have their own farm (one server in his own farm with all the roles). So 6 separate farms with one server in each.

        So the Timer Job should at least run once on each server and for now, it says that it never ran (“Non applicable” for the last execution time).

        It should run automatically till the end of the week.

        Jeff

        Reply

  6. What is surprising is that when i check the “Rotate machine keys in config DB” Timer Job on each server that was updated yesterday or last week with the october 2025 CUs, it says “Non applicable” for the last execution time for every server, as if the Timer Job never ran once.

    But on some servers (4 out of 6), i have machine keys displayed by your scripts.

    I do have to say that in july/august, when we installed the critical update, we also rotated the machine keys manually in the web.config of the web apps (maybe we forgot to do it on some servers).

    Jeff

    Reply

    1. Hi Jeff,
      manual key rotation before September CU used a different method and stored the info differently.
      Cheers,
      Stefan

      Reply

  7. Hi Stefan,

    It appears that I am running into issues with automatic Machine Key Rotation after installing October 2025 CU in a SharePoint 2016 environment.
    I plan to create a Microsoft support ticket. But decided to add a comment here as well – maybe you or others have faced similar issues.

    About the environment:
    – SharePoint 2016
    – Previous update level: August 2025 CU (September 2025 CU was skipped due to its issues)
    – 5 SP servers (2 of them are WFEs), 1 SQL server
    – The environment is mostly configured as per good practices. There are few areas for improvements, but the main things should be configured correctly – including the ones that would have been required to install September 2025 CU.

    No issues were observed directly after October 2025 CU was installed.
    But after the weekend (I assume – after the weekly Timer Job “Machine Key Rotation Job” was run) view state errors started to occur in UI when users where modifying document properties.

    In Windows Event Logs the following was showing up:

    Exception message: Unable to validate data.
    at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData)
    at System.Web.UI.Page.DecryptString(String s, Purpose purpose)

    at System.Web.Handlers.AssemblyResourceLoader.System.Web.IHttpHandler.ProcessRequest(HttpContext context)

    As it is a Production environment, then after initial investigation I decided to rotate Machine Keys manually (Set-SPMachineKey + IISRESET) in order to potentially have a temporary fix as soon as possible.
    It helped – the environment is working as expected now.
    But I am afraid that it may stop working again after next weekend when the Timer Job “Machine Key Rotation Job” runs. I could disable the Timer Job and it may help for now, but we are seeking for a permanent solution.

    Notes about the investigation so far:
    1) Another SharePoint 2016 environment (Test – 1 SP, 1 SQL) was also updated to October 2025 CU (previous update – August 2025 CU). It is configured as per good practices. No issues are observed in this environment. I am mentioning it as I will reference this environment below as well.
    2) Info: For all TimerService Instances “AllowServiceJobs” is “True” and “Status” is “Online”.
    3) About logs related to the Timer Job “Machine Key Rotation Job” – all expected entries are present in the Test environment. But in Production there is only the Timer Job start entry as expected. Further for the Timer Job and correlation token there are only several errors similar to the below. Note that the Timer Job completed successfully (when checking it in Central Admin).
    –
    SPLoggingLock held lock for 2391 milliseconds. Call stack:
    at Microsoft.SharePoint.Utilities.SPLoggingLockTag.Dispose()
    at Microsoft.SharePoint.Administration.SPPersistedObjectCollectionCache.InvalidateCache(SPConfigurationDatabase configDb, Guid objectId, Type objectType, SPPersistedObjectChangeEvent changeType, Boolean updateFileCache)
    at Microsoft.SharePoint.Administration.SPConfigurationDatabase.Microsoft.SharePoint.Administration.ISPPersistedStoreProvider.InvalidateCollectionCache(Guid objectId, Type objectType, SPPersistedObjectChangeEvent changeType)
    at Microsoft.SharePoint.Administration.SPPersistedObject.BaseUpdateCore(Boolean legacyUpdate)
    at Microsoft.SharePoint.Administration.SPEncryptedString.UpdateSecureStringValue(SecureString value)
    at Microsoft.SharePoint.Administration.SPSecureDBCredential.Update()
    at Microsoft.SharePoint.Administration.MachineKeyPersistedConfiguration.Persist(String validationKey, String decryptionKey, Guid webAppID, Boolean validateKeys)
    at Microsoft.SharePoint.Administration.SPMachineKeyRotationJobDefinition.Execute(Guid targetInstanceId)
    at Microsoft.SharePoint.Administration.SPTimerJobInvokeInternal.Invoke(SPJobDefinition jd, Guid targetInstanceId, Boolean isTimerService, Int32& result)
    at Microsoft.SharePoint.Administration.SPTimerJobInvoke.Invoke(TimerJobExecuteData& data, Int32& result)
    –
    SPLoggingLock waited 3359 milliseconds to acquire lock. Call stack:
    at Microsoft.SharePoint.Utilities.SPLoggingLockTag..ctor(UInt32 tagID, Object obj)
    at Microsoft.SharePoint.Administration.SPPersistedObjectCollectionCache.InvalidateCache(SPConfigurationDatabase configDb, Guid objectId, Type objectType, SPPersistedObjectChangeEvent changeType, Boolean updateFileCache)
    at Microsoft.SharePoint.Administration.SPConfigurationDatabase.Microsoft.SharePoint.Administration.ISPPersistedStoreProvider.InvalidateCollectionCache(Guid objectId, Type objectType, SPPersistedObjectChangeEvent changeType)
    at Microsoft.SharePoint.Administration.SPPersistedObject.BaseUpdateCore(Boolean legacyUpdate)
    at Microsoft.SharePoint.Administration.SPSecureDBCredential.Update()
    at Microsoft.SharePoint.Administration.MachineKeyPersistedConfiguration.Persist(String validationKey, String decryptionKey, Guid webAppID, Boolean validateKeys)
    at Microsoft.SharePoint.Administration.SPMachineKeyRotationJobDefinition.Execute(Guid targetInstanceId)
    at Microsoft.SharePoint.Administration.SPTimerJobInvokeInternal.Invoke(SPJobDefinition jd, Guid targetInstanceId, Boolean isTimerService, Int32& result)
    at Microsoft.SharePoint.Administration.SPTimerJobInvoke.Invoke(TimerJobExecuteData& data, Int32& result)
    –
    4) The hidden “Config Refresh” (“job-config-refresh”) Timer Job is present (accessible via PowerShell), but the property “LastRunTime” is displayed as “01.01.0001 00.00.00”. It is so in both Test and Production environments. Also, in none of the environments I see traces of the Timer Job running in SharePoint Logs. So, I am not sure if has been running. But note that the Test environment does not show any issues.
    5) The log entry “Successfully updated machine key” by “w3wp.exe” is present in the Test environment (that does not have issues). But it is not present in the Production environment (that has issues).
    6) Machine Keys in the Production environment (the problematic one) were present in the config cache (checked via your script) while the issue was happening. I am mentioning it because some persons in the comments have mentioned that Machine Keys didn’t exist for them in config cache.
    7) As described above, I used manual Machine Key rotation (Set-SPMachineKey + IISRESET) to temporarily fix the issue. I was expecting (based on your article) that after this activity Machine Keys in “web.config” files and config cache would be the same. However, they are different. I am not sure if this is as expected. But at least the issues are no longer present after this temporary workaround.
    8) I consider clearing SharePoint Configuration Cache (C:\ProgramData\Microsoft\SharePoint\Config\GUID) and running the automatic Machine Key rotation Timer Job again. Perhaps it works afterwards. However, as it is a world-wide Production environment, I could plan such test only during a weekend.

    If you have any comments or suggestions, then please let me know.

    Regards,
    Martins Grube

    Reply

    1. Hi Martins,
      please send me the case number after you have opened the ticket with Microsoft using “Contact the blog author” at the top right of this page.
      If I have sufficient bandwidth I will take ownership. Of course cannot promise it at this time. 🙂
      Cheers,
      Stefan

      Reply

  8. Hi Stefan,

    Would it be possible for you to produce a slightly different version of your script DumpConfigDBMachineKeys.ps1 that writes the machine keys in a file instead of in the console ?

    It would be interesting to create a scheduled task that dumps the machine keys every week to make sure that they are indeed rotated.

    Just a suggestion.

    Thanks !

    Jeff

    Reply

  9. Hi all,
    I am administering SP 2016 farm in single server topology.
    Firstly, I can not find any evidence of running TJ “job-config-refresh”. I am quite sure the timer service is ok and running, but in ULS there is no evidence, nor in CA or PowerShell (there is LastRunTime = N/A, or 01-Jan-01 00:00:00).
    I guess it is because this TJ is running under SharePoint Timer Service identity (account) which is Farm Account, and I am looking for evidences under another identity (FarmAdmin Account (under which was farm installed and is administered)).
    This is true also for TJ “job-config-machine-key-rotation”: it shows in CA LastRunTime = N/A, or 01-Jan-01 00:00:00 in PowerShell. But in ULS I can find both of messages: MachineKeyConfiguration_ password changed. Needs update… Successfully persisted MachineKeyConfiguration__ credential… that indicate machine keys are generated and updated in config db, for all of our web apps, and during all previous sunday mornings at 12:00:00 AM approximately.
    Can someone explain how these may/should be interpreted?
    With best wishes to everyone involved and many thanks to Stefan who is making great job supporting the community.

    Reply

    1. Little correction of myself: not all previous sundays, but all sundays after October 2025 CU installed.
      And small addition, on first sunday there were slightly different 3 messages (repeated for every web app) in ULS:

      Creating the SPPersistedObject. Updating object for the first time. SPEncryptedString Name=MachineKeyConfiguration_:MachineKeyConfiguration_

      Creating the SPPersistedObject. Updating object for the first time. SPSecureDBCredential Name=MachineKeyConfiguration_

      Successfully persisted MachineKeyConfiguration_ credential.

      Reply

    2. Hi eLKey,
      first of all, to verify if the machine keys have changed you can use my script which shows you the current keys:
      https://github.com/stefangossner/PowerShellScripts/blob/main/DumpConfigDBMachineKeys.ps1

      If you run this before executing the timerjob and (>30 seconds) after the timer job has run you should see that all machine keys have changed.

      If you see that these did not change, we need to look into the other messages in more detail.
      Cheers,
      Stefan

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.