This article describes software deployment and update procedures through SCCM.
Technical Advisory Committee
Article I. Supported Software
ITS Software Acquisition will maintain a list of supported software for SCCM application deployment and SCCM software updates. Customers may request new supported software by submitting a ticket to ITS-SOFTWARE. ITS-SOFTWARE will evaluate the suitability of supporting the software based on the following criteria and make a recommendation to the Desktop Management Technical Committee.
- Prevalence of software on campus
- Security impact of managing software
- Effort required to maintain software in SCCM
All patches from the Microsoft WSUS catalog are typically deployed.
Article II. Software Updates
Section 2.01 Normal Procedure
(a) Initiation
Microsoft typically releases software updates on the 2nd Tuesday of each month, also known as Patch Tuesday. The SCCM software manager will package third-party updates in SCUP and make these available on the 2nd Tuesday of the month. Updates released at any other time, will be held until the next Patch Tuesday, unless they meet the requirements for an out-of-band deployment.
(b) Evaluation
Software updates will be deployed to the “Testing” group when they are released. The testing period is 1 week, starting on the 2nd Tuesday and ending on the 3rd Tuesday. Members of the “testing” group should install the updates and provide feedback as soon as possible.
(c) Approval
Updates will be approved for installation on the 3rd Tuesday if no major problems have been reported. The “early” group will have a deadline of 2:00 AM on the 3rd Tuesday. The “normal” group will have updates become available at 2:00 AM on the 3rd Tuesday with a deadline of 2:00 AM on the 4th Tuesday. This week of availability allows updates to be scheduled by the end user using the business hours functionality in Software Center.
If problems are reported, the update may be held back temporarily until solutions or workarounds are found, up to 90 days. Due to the cumulative nature of most Microsoft updates, it is not practical to hold back updates indefinitely. Requests to hold back an update will be approved by the Desktop Management Technical Committee.
(d) Controlling Update Deployments
Software update deployment can be managed via the following techniques.
(i) Maintenance Windows
Maintenance windows specify that updates (and required reboots) can only occur during the specified window. When a maintenance window is present, a computer will install updates at the next suitable maintenance window after the deadline. The maintenance window must be long enough for all the scheduled updates to complete (typically 5 minutes per update).
Maintenance windows are configured in the collection properties in SCCM by SCCM administrators.
(ii) Business Hours
Business hours allows the end user to specify the times that they use their computer. When enabled, SCCM will automatically schedule the updates to install outside of the specified business hours. Business hours are in effect after the updates are available but before the deadline. When the deadline is reached, the updates will install immediately, regardless of the business hours configuration.
(iii) Customer Managed Deployments
SCCM administrators may elect to opt-out of the “campus deployment” and create their own deployments. This allows a custom availability and deadline schedule.
(iv) Grace Period
The software update grace period allows users to delay the installation of updates when the deadline is reached, for a preconfigured amount of time. For example, many mobile devices are often offline for long periods of time. If a device has missed its deadline and a user turns the device on to give a presentation, it would immediately begin installing updates. The grace period allows the user to delay the installation.
By default, the grace period is 8 hours. Departments may override this as desired. Each deployment must also be configured to enable the grace period. This setting is located in the scheduling tab of the deployment. Deployments of software updates to campus will be configured to allow the grace period.
(e) Controlling Reboots
If no one is logged on to the computer, it will reboot immediately after finishing installing updates that require a reboot. If a user is logged on, a prompt will indicate that the system requires a reboot and will begin a count down. By default, the countdown will be 4 hours and the notification cannot be hidden for the final 90 minutes. These parameters can be customized upon request.
(f) Software Update Reporting
A couple of tools are available for monitoring software update installation and compliance.
1) Splunk Software Update Compliance Dashboard
To use the Splunk Update Compliance Dashboard, select your department and an update deployment and click submit. It will provide a list of compliant, in progress, failed, and unknown systems.
2) Splunk Maintenance Windows Dashboard
The Splunk Windows Dashboard allows you to easily view all the maintenance windows that apply to a system and its next maintenance window, as well as a list of systems with upcoming maintenance windows in the next 24 hours.
3) SCCM Reports
This report is available in the SCCM Console under Monitoring\Reporting\Reports\Software Updates – A Compliance\Compliance 1 – Overall Compliance. Run the report, choose an update group and collection and it will provide a drilldown report of compliant, non-compliant, and unknow systems.
Section 2.02 Out-of-Band Procedure
Out-of-band updates are updates are software updates are released outside of the normal schedule and address a major security or functionality issue. Most updates released outside the normal schedule will be held until the next Patch Tuesday.
Out-of-band updates must meet the following criteria:
Address a business critical issue
Urgent, i.e. waiting until Patch Tuesday would likely result in a major security breach or loss of productivity
(a) Initiation
Out-of-band updates are typically initiated by the ITS Information Security team, but may also originate from SCCM administrators or application owners.
(b) Evaluation
Updates that meet the out-of-band criteria will be deployed immediately to the “test” group. Members of the test group will provide feedback about any problems and about the suitability of the patch for an out-of-band deployment.
(c) Approval
The Desktop Management Technical Committee will have a conference call to discuss the update and the results of testing. Updates will be approved for out-of-band deployment with a majority vote. Updates may also be approved by the Chief Information Security Office (CISO).
(d) Controlling Update Deployments
Out-of-band update deployments may be managed like section 1.01(d) above. Out-of-band updates may bypass maintenance windows, but only in the most extreme circumstances at the request of the CISO.
Upon request, departments may opt-out systems from the out-of-band deployment. Because out-of-band updates are often deployed on an abbreviated schedule, it is important that opt-out requests be submitted as quickly as possible.
Section 2.03 Firewall Pre✓
The Firewall Pre✓program is an opt-in program to enable quicker turn around for firewall requests. Computers that are opted-in, will be configured as follows:
- Include in the “early” patch group
- Create a backstop deployment group that ignores maintenance windows if computers are non-compliant at 21 days.
- Deploy Adobe Flash, Adobe Reader, and Java updates out-of-band
- Compliance reporting will be available in Splunk
- Compliance will be verified using Qualys, either by regular scans or by installation of the Qualys software agent to provide continuous vulnerability assessment without the need for scans or reconfiguration
To opt-in, created a group called “<DEPT>_Firewall Precheck Opt-In” and add the desired computers to the group. Send a ticket to ITS-SYSTEMS requesting that your group be configured for Firewall Precheck.
Section 2.04 Catchup Deployment Procedure
A catchup deployment will be created each month on the 1st Tuesday with the purpose of redeploying older updates that are still non-compliant on SCCM clients. The catchup deployment will include any update, that is required by a SCCM client, with a release date on or before Patch Tuesday 2 months prior. The following categories will be included
(a) Categories
- Critical Updates
- Security Updates
- Updates
- Update Rollups
- Service Packs
Microsoft updates will be deployed to both client and server operating systems. Third-party updates will only be deployed to client operating systems.
(b) Scope
Deployments will be scoped to the All Desktop and Server Clients collection, which includes all systems with a SCCM client. This deployment will apply to systems that have been excluded from the normal update deployments. Departments using alternate update systems or procedures will only be affected if their systems are still non-compliant on the catchup deployment deadline.
Any exceptions must receive approval from ITS Information Security.
(c) Schedule
Catchup deployments will be created every 1stTuesday of the month. The updates will be made available immediately. The deadline will be configured for the 3rdTuesday at 2:00 AM.
(d) Maintenance Windows
The catchup deployment will be configured to ignore maintenance windows for update installation. The catchup deployment will be configured to observe maintenance windows for reboots, i.e. systems will not be automatically rebooted unless they are in a maintenance window.
(e) WSUS Deployment
If the catchup deployment via SCCM is found to be ineffective, updates may be deployed via WSUS using the same criteria. Customers can control how WSUS installs updates via Group Policy settings for Windows Update.
Section 2.05 Update Deployment Cleanup
Software update deployment groups (SUGs) and deployment packages will be deprovisioned 6 months after their creation. All OS deployment images must be patched to within 6 months to ensure that newly imaged systems are fully patched.
Article III. Windows 10 Servicing
Section 3.01 Definitions
Insider Preview Branch – These are previews of new Windows 10 releases intended for testing and small pilots. These releases are suitable for test labs and secondary devices.
Current Branch (CB) aka “release ready” – The initial release of a new Windows 10 version. This is suitable for pilot deployments, IT staff, and early adopters
Current Branch for Business (CBB) aka “business ready” – After about 4 months, the current branch is promoted to the current branch for business. This milestone indicates the release is suitable for general deployment. At any given time, there will be 1 CBB.
Section 3.02 Release Cadence
Microsoft releases a new current branch about every 8 months. The current branch becomes the current branch for business after about 4 months. Microsoft provides primary support for the most recent 2 versions. Support for the 3rd version back ends about 60 days after the current branch becomes the current branch for business. It is important to promptly service Windows 10 because unsupported versions will no longer receive security updates.
Section 3.03 Deployment Mechanism
Windows 10 will primarily be servicing via the SCCM Windows 10 Servicing feature. This works similarly to software update deployment. Servicing upgrades will appear in software center as software updates. There are 3 phases to servicing upgrades:
- The update is installed in Software Center. The system is usable during this phase. When complete, the device will need to be rebooted to continue the installation. The reboot can be delayed according to the device’s SCCM client settings policy.
- Post reboot steps will begin during Windows bootup. This is similar to the post reboot procedure for software updates, but will take longer, about 25 minutes. The computer will be unusable until this completes. A message will be displayed that Windows is applying updates.
- Post logon steps will be performed after the user logs on. The user will be displayed the Windows 10 first logon animation. The computer will be unusable until this completes, about 5 minutes.
Section 3.04 Deployment Strategy
Windows 10 servicing upgrades will be deployed in waves, beginning with pilot rings and progressing to general deployment rings. The pilot phase will begin when a new current release is available. The broad deployment phase will begin when a new current branch for business is available. The broad deployment deadlines will be aligned with normal monthly patching windows. However, servicing upgrades will be available in the Software Center to the broad deployment group for the entirety of the broad deployment phase to allow users the opportunity to install the upgrade at an opportune time before the deadline is reached.
Pilot Phase– Opt-in deployments
Pilot Ring IT – Opt-in consisting of IT staff test/secondary devices. We do not recommend adding primary devices, as this is the first test deployment and problems are likely.
Pilot Ring QA – Opt-in for testing critical applications
Pilot Ring Early Adopters – Opt-in for general deployment to early adopters. We highly encourage all remaining IT devices to be a part of this release.
Broad Deployment Phase– Ring memberships will be created randomly from devices that are not in another ring
Broad Deployment Ring I
Broad Deployment Ring II
Broad Deployment Ring III
Broad Deployment Ring IV
Deferred Deployment Phase
Deferred Deployment Ring – Opt-in for devices that should be upgraded last
Section 3.05 Timeline
Timelines are approximate and subject to change based on the University calendar. Upgrades will be blacked out during certain critical times as appropriate, such as exams, semester beginnings, winter break, etc.
Pilot Phase– Available immediately following current branch release
Pilot Ring IT – Deadline 2 weeks following current branch release
Pilot Ring QA – Deadline 4 weeks following current branch release
Pilot Ring Early Adopters – Deadline 2 months following current branch release
Broad Deployment Phase– Available immediately following current branch for business release
Broad Deployment Ring I – Deadline 1 month after CBB release, aligning with monthly patches
Broad Deployment Ring II – Deadline 2 months after CBB release, aligning with monthly patches
Broad Deployment Ring III – Deadline 3 months after CBB release, aligning with monthly patches
Broad Deployment Ring IV – Deadline 4 months after CBB release, aligning with monthly patches
Deferred Deployment Phase– Available 5 months after CBB release
Deferred Deployment Ring – Deadline 6 months after CBB release, aligning with monthly patches
Section 3.06 Communication
Communications will be sent to the SysAdmins Team announcing upgrade schedules. Departments are responsible for providing appropriate communication to end-users about servicing upgrades.
Article IV. Collection Management
Section 4.01 Collection Folders
SCCM collection folders are useful for organizing collections. While collections are only visible if they are within the administrator’s security scope, folders are visible to all. Also, a permissions quirk in SCCM requires that permissions to create folders be granted in order to allow administrators to move objects between folders. As a result, it is important to carefully manage folders in SCCM.
(a) Device Collection Folders
- Antimalware Policies – Dedicated collections for deploying antimalware policies
- Applications – Dedicated collections for deploying applications
- Client Health – Collections for monitoring client health (obsolete or out of date clients)
- Client Settings – Dedicated collections for deploying client settings
- Compliance – Dedicated collections for deploying compliance baselines
- Departments – Folder to allow departments to build out their own folder hierarchies. Each department may create one folder matching their OU name and may create any folders they want underneath that folder.
- Identity Finder – Collections for monitoring Identity Finder deployment compliance
- Operating Systems – Dedicated collections for deploying operating system task sequences
- Maintenance Windows – Dedicated collections for configure task sequence maintenance windows
- OU Memberships – Collections the represent OU memberships
- Packages – Dedicated collections for deploying packages
- Software Updates – Top level collections used to configure software update deployment
- Maintenance Windows – Dedicated maintenance windows for configuration software update maintenance windows
- Microsoft Updates – Dedicated collections for deploying Microsoft updates
- Third-Party Updates – Dedicated collections for deploying third-party updates
- Warranty Status – Collections for managing warranty status compliance
- Windows 10 Servicing – Dedicated collections for deploying Windows 10 upgrades
(b) User Collection Folders
- Applications – Dedicated collections for publishing applications
- Client Settings – Dedicated collections for deploying client settings
- Departments – Folder to allow departments to build out their own folder hierarchies. Each department may create one folder matching their OU name and may create any folders they want underneath that folder.
Article V. SCCM Servicing
Microsoft releases new SCCM current branches about 3 times per year. It is important that we update SCCM promptly because each version is supported for only 12 months following its release. In addition to security fixes and new features, these updates are often necessary to support the latest release of Windows 10.
When a new SCCM update is released, it will be installed in a test environment as soon as possible for review and evaluation. Unless any major issues are discovered, the production environment will be scheduled for upgrade within 2 months. Typically, the ADK, which includes the Windows PE boot environment and USMT, will be upgraded at the same times that SCCM is updated. SCCM client upgrades will begin 2 weeks after the server is upgraded, following a test deployment to the “testing” group. Server upgrades will begin 6 weeks after the server upgrade. In both cases, the upgrades will be configured to occur at random times over a 7 day period.
Communication about upgrades will be made to the SysAdmins Team and the ITS Change Management system.
Article VI. Client Configuration
Section 6.01 Client Cache
SCCM clients maintain a cache on the local C drive that is used to state files for software updates, upgrades, applications, etc. By default, this cache is 5 GB. The SCCM client only uses (and encumbers) the amount of the cache that is actually in use. We’ve found over time that 5 GB is not big enough to accommodate some larger deployments. Some applications deployments are 20 GB and new computers often install many applications and updates when they first come online. In these cases, deployments will fail.
SCCM client cache size will be set to 50% of the free space on the C drive (or whatever drive the cache is located on) or 40 GB, whichever is smaller. The SCCM client will check the amount of free space daily and adjust the cache size to ensure SCCM does not fill up the disk.
Article VII. Operating System Deployment
Section 7.01 Operating System Images
ITS will make available the latest 2 versions of Windows 10 in the form of Operating System Images and Operating System Upgrade Packages. Both will be scoped to “Everyone”.
When a new version of Windows 10 is made available a notification of its availability will be sent to the SysAdmins team SCCM channel. Admins will then have 2 weeks to update existing task sequences to use the two most current images. Any image or update package 3 versions back will be scheduled for removal two weeks after the announcement. A list of Task Sequences that are using the image scheduled for removal will be sent to the SysAdmins team SCCM channel at the time of the announcement.
Section 7.02 Image Patching
Updates will be approved for installation following the same schedule mentioned in Article II Section 2.01 c. Updates will be scheduled for installation on the 3rd Tuesday at 2:00am and will be applied to both Operating System Images and Operating System Upgrade Packages. Images will be stored at \\ad.unc.edu\sccm\unc\Content\OSD\OSImages. A Backup of the OS images will be stored for 1 month. ITS will perform a test OS deployment for both of the currently maintained OS images following each patch cycle.
Section 7.03 Driver Support
Drivers will be added to SCCM for supported MS Surfaces and Lenovo CCI models when new models become available. CCI models will be supported for 5 years. Network and storage drivers for other models will be added upon request. Anyone with frequent driver needs outside of supported models should make a request through the DTCM or consider a 3rd party tool such as UIU.
Section 7.04 Boot Images & Boot Media
A 64-bit boot image will be made available and will include support for Microsoft Deployment Toolkit and PXE booting. Additional support and features will be added to the boot image as approved by the DMTC. Boot media will be created using the most recent boot image and made available to campus.
Section 7.05 Task Sequences
ITS will create and make available a generic task sequence for the latest 2 versions of Windows 10 images. It will be scoped to “Everyone” and a deployment will be created making the task sequence available to “All Unknown Computers” using boot media or PXE. Departments are permitted to copy or create their own task sequences but are asked to limit the number of task sequences they deploy to “All Unknown Computers” to 2 per department.