Fixing Veeam Backup v8 Periodic Scheduling

Two weeks ago, I upgraded my employer’s Veeam Backup and Replication infrastructure from Version 7 to Version 8 with Update 1. Most aspects of the upgrade went well, and Veeam BR is still a good product. That said, some recent changes to the scheduling components introduced some challenges.

Backup Schedule Options

When you create backup jobs, you typically define a schedule regarding when you want those jobs to run. Veeam BR offers several options, the most common are:

• Daily: To run the job at specific time daily
• Monthly: To run the job once a month on specific days
• Periodically: To run the job repeatedly throughout a day with a set time interval

In my current environment, we prefer to schedule backup jobs to run Periodically. Veeam BR 8 made several changes to how periodic backup jobs are scheduled — changes that offer fewer options and produce undesired results. This post will dive into those concerns.

Hospital Objectives

I work for a hospital. Since systems are always in use, there’s rarely a good time to perform backups. For this reason, we perform backups all the time; during the day and evening.

We’re less concerned about when backup run. Our objectives are:

• Have backups run X number of times per day
• Have backups run quickly (e.g. a short duration of time)
• Stagger backups so that only a few run at a time

With Veeam BR 7, we met these objectives by using period schedules and a coin.

For example, let’s say we had Tier 1 (backup about every 12 hours), Tier 2 (backup about every 24 hours), and Tier 3 (backup about every 48 hours) backup jobs. When we created a new Tier 1 backup job, we’d do the following:

If it was a Tier 1 backup job…

• Flip a coin
• If tails, configure the job to backup every 11 hours
• If heads, configure the job to backup every 13 hours

If it was a Tier 2 backup job…

• Flip a coin
• If tails, configure the job to backup every 23 hours
• If heads, configure the job to backup every 25 hours

If it was a Tier 3 backup job…

• Flip a coin
• If tails, configure the job to backup every 59 hours
• If heads, configure the job to backup every 61 hours

Using this simple workflow, half of our jobs would slightly shift to occur slightly less than once a day or half-day, while others would shift to occur slightly greater than once a day or half-day. The result was we did not have to micromanage when each backup job was to occur or even when we created the backup job.

By slightly shifting periodic backup schedules, rarely would many jobs attempt to run at the same time. Using this method we were meeting our service level agreements, our backup jobs were evenly staggered, and system resources were not constrained.

Problem 1: All Periodic Jobs Run at Midnight

That all changed with Veeam Backup and Replication version 8. The biggest change that we dislike is how Veeam handles periodic backup jobs that are scheduled to next run sometime in the following day.

Here’s the documented change:

Veeam Backup & Replication always start counting defined intervals from 12:00 AM. For example, if you configure to run a job with a 4-hour interval, the job will start at 12:00 AM, 4:00 AM, 8:00 AM, 12:00 PM, 4:00 PM and so on.

The result of this change is that all periodic backup jobs will attempt to run between midnight and 1 am every day, which creates a heavy demand on infrastructure resources and causes backups to take longer to complete.

For example, if a job was scheduled to run every 5 hours and last ran at 10 pm, one would expect it to next run at 3 am. Not with Veeam BR 8. With Veeam BR 8, it will run the job at midnight instead, because it resets the clock each day.

Problem 2: Cannot schedule beyond 24-hours

The second issue we have with the Veeam BR 8 scheduler is that you may no longer schedule backup jobs to periodically occur for a time-period greater than 24 hours.

Not all jobs need to run daily. For some, the rate of change is so low that performing backups every three or five days would be a better use of resources. As for a jobs that were scheduled to run every 24+ hours, they all began running shortly after midnight.

Problem 3: Fewer scheduling options

Prior to Veeam BR 8, you could schedule backup jobs to occur every 5, 7, 11, 13, 23, 25, or 33 hours (among many more options). Veeam BR 8, however, limits you to values that evenly divide into 24 (e.g. 1, 2, 3, 4, 6, 8, 12, 24). This limits our ability to periodically schedule jobs to sift/stagger so that they do not run at the same time each day.

Instead, we have to schedule these values in minutes; calculating that an 11 hour job should run every 660 minutes. Not a big deal, but an inconvenience. To address this problem, I wrote a script to Randomize your Veeam backups via PowerShell.

The problem, of course, is that all jobs still run at midnight, even if they were not scheduled to run until several hours later.

Using Veeam PowerShell and Windows Task Scheduler to fix Veeam BR 8 Periodic Schedules

Veeam says that all of the periodic scheduling problems that we are experiencing with Veeam Backup and Replication version 8 are by design (e.g. are not bugs/flaws with the application).

Since we still want to use periodic schedules, I wrote the following Veeam PowerShell script to correctly calculate the Next Run time even if it occurs the following day. You’ll need to also use Windows Task Scheduler to have this script automatically run at 11:55 pm on each of your Veeam backup repositories.

Here is a plain text Version 1 of this script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

 # Fix Veeam Backups Next Run, a Veeam PowerShell script (Version 1)
 # This script will properly set the Next Run schedule when entering a new day so that all jobs don't run between midnight and 1 am
 # A Windows Task Scheduler must be configured to run this script on each Veeam Backup Repository server -- running at 11:55 pm each evening
 # Run this on a Veeam Backup and Replication version 8 update 1 server
 # You could even copy and paste it into Veeam Backup and Replication > Menu > PowerShell
 # Written by Jason Pearce of jasonpearce.com in March 2015
 # Use at your own risk, freely share, and comment on improvements
 
 # BEGIN Script ###############################
 
 # Enable Veeam PowerShell Snapin
 Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue
 
 # BEGIN variables ############################
 
 # Get All, Some, or One backup jobs.
 # You MUST uncomment and modify one of these lines to target one or more backup jobs.
 # Only one line should begin with "$jobs =". These are just four helpful examples.
 #By-All# $jobs = Get-VBRJob | Where-Object { $_.IsBackup -eq $true -and $_.IsScheduleEnabled -eq $true } | Sort Name
 #By-PREFIX# $jobs = Get-VBRJob -Name "Test-*" | Where-Object { $_.IsBackup -eq $true -and $_.IsScheduleEnabled -eq $true } | Sort Name
 #By-LIST# $jobs = Get-VBRJob -Name "Test-1","Test-2" | Where-Object { $_.IsBackup -eq $true -and $_.IsScheduleEnabled -eq $true } | Sort Name
 #By-NAME# $jobs = Get-VBRJob -Name "Test-Job" | Where-Object { $_.IsBackup -eq $true -and $_.IsScheduleEnabled -eq $true } | Sort Name
 
 # END variables ##############################
 
 # BEGIN foreach loop #########################
 
 # Make the following changes to all backup jobs
 foreach ($job in $jobs) {
 
 # BEGIN Job Schedule Options #############
 
 # Get Current Job Schedule Options
 $currentJobScheduleOptions = Get-VBRJobScheduleOptions -Job $job
 
 # Create a new job schedule 
 $newJobScheduleOptions = New-VBRJobScheduleOptions
 
 # Enable Periodically Schedule
 $newJobScheduleOptions.OptionsPeriodically.Enabled = $currentJobScheduleOptions.OptionsPeriodically.Enabled
 
 # Get Current Periodic Schedule Kind (0 = Hours, 1 = Minutes)
 $newJobScheduleOptions.OptionsPeriodically.Kind = $currentJobScheduleOptions.OptionsPeriodically.Kind
 
 # Get Current Other Schedules
 $newJobScheduleOptions.OptionsDaily.Enabled = $currentJobScheduleOptions.OptionsDaily.Enabled
 $newJobScheduleOptions.OptionsMonthly.Enabled = $currentJobScheduleOptions.OptionsMonthly.Enabled
 $newJobScheduleOptions.OptionsContinuous.Enabled = $currentJobScheduleOptions.OptionsContinuous.Enabled
 
 # Get Current FullPeriod offset
 $newJobScheduleOptions.OptionsPeriodically.FullPeriod = $currentJobScheduleOptions.OptionsPeriodically.FullPeriod
 
 # Get Current HourlyOffset
 $newJobScheduleOptions.OptionsPeriodically.HourlyOffset = $currentJobScheduleOptions.OptionsPeriodically.HourlyOffset
 
 # Get Current RetryTimeout
 $newJobScheduleOptions.RetryTimeout = $currentJobScheduleOptions.RetryTimeout
 
 # Do some math to calculate the desired NextRun value by adding FullPeriod to LastRun
 $newJobScheduleOptions.LatestRun = $currentJobScheduleOptions.LatestRun
 $newJobScheduleOptions.NextRun = ($currentJobScheduleOptions.LatestRun).AddMinutes($currentJobScheduleOptions.OptionsPeriodically.FullPeriod)
 
 # Apply the new job schedule
 Set-VBRJobScheduleOptions -Job $job -Options $newJobScheduleOptions
 
 # END Job Schedule Options ###############
 
 # Report which jobs received these changes
 Write-Host "Changed settings for" $job.Name
 }
 # END foreach loop ###########################
 # END script #################################

And here’s how I created a Schedule Task, which is done on each Veeam BR 8 Repository Server.

• Copy script to C:\Scripts\fix-veeam-backup-nextrun-for-new-days-by-jason-pearce.ps1
• Windows Task Scheduler > Create Task
• General > Name > Veeam Fix Next Run
• General > Description: With Veeam 8, all periodic backups run at midnight. This script changes the Next Run date/time so that it is not reset at midnight and will continue into the next day.
• General > Use the following user account > I use my Veeam Service Account, which has local admin rights
• General > Run with highest privileges
• Triggers > New
• Triggers > Begin the task > On a schedule
• Triggers > Start > Daily at 11:55 pm
• Triggers > Repeat task every 10 minutes for a duration of 15 minutes
• Triggers > Enabled
• Actions > New
• Actions > Action > Start a program
• Actions > Program > PowerShell
• Actions > Add arguments > C:\Scripts\fix-veeam-backup-nextrun-for-new-days-by-jason-pearce.ps1
• Conditions/Settings/History no change, use defaults

Perhaps Veeam will change its mind and revert back to the more flexible and less problematic periodic scheduler. In the meantime, I hope my script above helps.