Solving the extended install time for SPSE CUs

When comparing the installation time for CUs for SharePoint Server 2019 and SharePoint Server Subscription Edition there is a significant difference.
E.g. on my test machines installing a SharePoint Server 2019 fix takes roughly 20 minutes compared to nearly an hour for a similar SharePoint Server Subscription Edition fix.

The reason for this extended install time is the fact that for SharePoint Server 2019 fixes the installer stops all relevant SharePoint and IIS Windows Services before invoking the fix installation while the services fail to be stopped for SharePoint Server Subscription Edition fixes. The root cause here is that the method used to stop the services no longer works with SharePoint Server Subscription. This leads to performance problems when the installer tries to update assemblies in the global assembly cache due files being in use.

Solution

To improve the performance the issue the relevant Windows services need to be stopped before the fix is applied and restarted afterwards.
Here is the list of services to be stopped before the fix is applied. The sequence is important to ensure that the services remain stopped and are not automatically restarted:

Server Role Service
All SharePoint Timer Service (SPTimerV4)
All SharePoint Tracing Service (SPTraceV4)
All SharePoint Administration (SPAdminV4)
All World Wide Web (W3SVC)
Search SharePoint Server Search (OSearch16)
Search SharePoint Search Host Controller Service (SPSearchHostController)
Distributed Cache SharePoint Caching Service (SPCache)

The reverse sequence needs to be applied when restarting the services after the SharePoint CU has been installed.

Important: Extra care needs to be taken for the Distributed Cache when using zero-downtime-patching as a graceful shutdown of the Distributed Cache host is required to guarantee that the cache data is replicated to other Distributed Cache servers still running in the farm before the service is stopped.

Solution Script

The above listed steps of stopping/restarting the services can be automated using a script. For your convenience I have created a sample script which performs the required operations:
https://github.com/stefangossner/Install-SPSE_Fix/blob/main/Install-SPSE_Fix.ps1

The script has two parameters:

  1. CULocation (mandatory): Here you pass in the location of the CU you are planning to install. E.g. C:\temp\uber-subscription-kb5002560-fullfile-x64-glb.exe
  2. ShouldGracefulStopDCache (optional): pass in $true if a graceful shutdown of the distributed cache service on the current server should be attempted,

E.g: to apply February 2024 SharePoint CU which was previously downloaded to the temp directory on a server hosting an instance of the Distributed Cache in a farm with multiple Distributed Cache Hosts where zero downtime patching is being performed:

Install-SPSE_Fix.ps1 -CULocation C:\temp\uber-subscription-kb5002560-fullfile-x64-glb.exe -ShouldGracefulStopDCache $true

Without zero downtime patching the following command should be used:

Install-SPSE_Fix.ps1 -CULocation C:\temp\uber-subscription-kb5002560-fullfile-x64-glb.exe

10 Comments


  1. Many thanks for sharing, Stefan! I have some “déjà vue” with SP2013 patching when I see the approach. 🙂

    Reply

    1. Indeed – although the root cause there was different. In SP2013 the time delay occurred due to the large amount of packages we installed – and between each package we stopped and restarted the services which took a long time. The famous “Russ Maxwell script” disabled the services preventing them from starting and stopping leading to performance gains.
      Here the problem is that the services are not stopped in the first place.

      Reply

  2. We usually restart the server for extra precaution after the CU patch. So I guess we would still start the services in reverse order before restarting the server then. This is a good find and will quicken the patching process.

    Reply

  3. Hi Stefan!
    I need your advice on how to reduce the running time of Configuration Wizard during CU installation. I have a very large Sharepoint 2019 farm (4 servers per role), 4 SQL clusters and 1200 content databases. As a result, the Configuration Wizard on the main server (on which the CA is installed) runs for about 10 hours + about 15-20 minutes on all other servers. The total time, taking into account the installation of the files themselves, takes 15-16 hours.

    Naturally, before installing the CU, I stop the search service via Suspend-SPEnterpriseSearchServiceApplication.

    Is there any way to speed up the Configuration Wizard in this farm configuration?

    Reply

    1. Hi Evgeny,
      I’m pretty sure the majority of time is spent on the upgrading the 1200 content databases.
      But this is something you don’t have to do in the config wizard!
      My suggestion would be – before starting the config wizard, start 10+ powershell windows and run upgrade-spcontentdatabase in parallel for mulitple databases.
      You can even create a Powershell script which launches multiple powershell instances in parallel to automate this.
      After the content databases are upgraded you can then launch the config wizard and it will only have to upgrade the config db and the service application databases and perform the other local machine tasks.
      Cheers,
      Stefan

      Reply

      1. Stefan, thanks a lot!

        Yes, majority of time is spent on the upgrading the 1200 content databases – it’s information appers in the Diagnostioc Logs.
        I’ll try running multiple PS-sessions next time.

        Reply

  4. Hi Stefan, thanks for sharing. As usual you’re a life saver. I want to ask that, this situation will be fixed with future updates?

    Reply

    1. Hi Hasan,
      a fix request is currently in the backlog but we do not have information when it will be picked up.
      Cheers,
      Stefan

      Reply

  5. Hi Stefan,

    When using the new script on our 2 AP/3 WFE SPSE farm it has greatly improved the install time for our app servers, non-user access, crawl servers but still seeing over an hour or hour and 1/2 to install same on WFE servers. We stop user traffic via the Load balancer however we have noticed several suspended node runner processes where the PIDs correspond to the Restart Manager event log errors .
    ‘Unknown’ (pid 16848) cannot be restarted – Application SID does not match Conductor SID..

    If stopping the OSearch16 service isn’t killing those noderunner processes is there another way to do that before installing updates?

    Reply

    1. Hi Allison,

      i just suspend the Search Service Application, run the installation and after the complete patching process am starting it again.
      $ssa = Get-SPEnterpriseSearchServiceApplication -Identity “Your Search Service Application”
      Suspend-SPEnterpriseSearchServiceApplication -Identity $ssa
      Running the patching process and after that
      Resume-SPEnterpriseSearchServiceApplication -Identity $ssa

      Helped for me. It took just 30 minutes on my Single Server (with all the search components as well)

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.