SQL Server log truncation fails with Veeam Backup & Replication

When using Veeam Backup & Replication 9.5 to backup SQL Server 2012 databases the transaction log was failing to be truncated for one database.  The database was using the full recovery model and the user account specified in Veeam’s Application Aware Processing had the required permissions and was successfully truncating the log of other databases on the same SQL instance.  The warning is displayed in Veeam as below.

SQLLog1

Veeam logs additional information in the VeeamGuestHelper log in C:\ProgramData\Veeam\Backup\VeeamGuestHelper_DATE.log.

SQLLog2

To resolve the error I added the following registry keys in the two locations below.  In my case the full registry path didn’t exist under Wow6432Node so I had to create it manually.

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\VeeaM\Veeam Backup and Replication\

HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication\

Name: SqlExecTimeout
Type: DWORD
Value: 600

Name: SqlLogBackupTimeout
Type: DWORD
Value: 3600

Name: SqlConnectionTimeout
Type: DWORD
Value: 300

Advertisements

SharePoint Service Pack 1 – Installation of this package failed

When installing SharePoint 2013 service pack 1 on a SharePoint farm, the extraction of the service pack kept failing on one web front end.

When clicking on the officeserversp2013-kb2880552-fullfile-x64-en-us.exe file, the User Account Control dialog appeared, then when I continued to extract the files, the process failed with “The installation of this package failed”.

SP1-Error-1

Looking the service pack log file showed that some files were successfully extracted before the error.

SP1-Error-2

In order to work around the error on this server I found I needed to open an administrative command prompt and then run the service pack.

 

Azure IaaS Cost Optimisation

When talking to people about Microsoft Azure one of the biggest concerns is cost management and how to avoid runaway spend.  In this post I’ll share my tips for optimising Azure expenditure.  In my organisation I’ve used these tips to reduce Azure expenditure by around 50%.

Right size CPU

If you use the Azure portal to create a Windows Server 2016 VM, the default size as of Nov 18 is a DS1 v2 which costs £72.36 a month in West Europe.  If you don’t take the time to check what’s the most appropriate VM series and size you could be paying more than you need to, or under sizing your VM.  If you’re running a low end production server, or dev/test server, you could use a B series VM such as a B2S, which costs £35.26 a month.  In this case you’ve instantly saved ~50%.

Information on VM sizes is available here.  If you’re sizing for a workload that requires high storage throughput take care to check the disk IOPs and throughput measurements.  You don’t want to under size your VM, but you don’t want to over pay either.

When sizing for high CPU performance check the compute benchmark scores as the CPU performance varies between VM series.

If you want to resize a VM, Azure Automation is a great way, see here for example code in PowerShell.

Right size disk

If you use the Azure portal to create a Windows Server 2016 VM, the default disk type is premium SSD.  A 128 GB  premium SSD disk is £16.16 a month.  If you don’t require a single instance SLA, or high performance, you could use a standard SSD disk at £7.15 a month + transaction costs.  Again, this is a ~50% saving.

Information on disk type is available here.  If you chose to use premium storage for performance reason be sure your workload can utilise the disk fully.  Information on how to monitor this is available here.

Managed disks allow you to switch between the disk types, e.g. standard HDD, standard SSD and premium SSD.  You can use Azure Automation to automate the change, see here for example code.

Right time

Once you have the right VM and disk types think about when you need to run the VM.  If you have a dev/test workload you could use automation to only run the VM during the working day, 08:00-18:00.  By running the VM for 10 hours a day rather than 24 you can save significantly on the compute costs.  Note that you continue pay storage costs when the VM is turned off.

From the VM blade in the Azure portal you can configure auto-shutdown to turn of the VM at a set time each day.  For more advanced start/stop scenarios look for PowerShell solutions such as this.

Reserved Instances

Reserved Virtual Machine Instances enable you to make significant savings on compute costs.  The saving varies by VM series and the duration of the reservation, but 1 year reservation for a D2s v3 VM is 36%, and for a F2s v2 it’s 24%, both significant savings.

Clean up

Don’t waste money on resources you no longer need.  As mentioned earlier, even if you turn off a VM you still pay storage costs.  If you delete a VM the disk is left behind, again costing money.  Azure Automation is a great way to identify and remove unattached disks, see here for example code.  Don’t forget to look for other resources that are no longer used, e.g. unattached public IP addresses.

Tagging

When you create a resource group, add appropriate tags to enable you to identify the system, owner and role for the resources contained with it.  Use automation to cascade the tags to all resources within the group; example code is available here.  You can then use Power BI as I describe here to create detailed reports on usage by tag, resource type etc.  Combining tagging with Power BI is a very powerful combination.

SharePoint 2013 – Sorry, something went wrong

Following some work on SharePoint, users were encountering the error “Sorry, something went wrong.  The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)”

SP-Content-Expired-0

Searching the web uncovered suggestions to change the claims token timeout value, reducing it from its default value of 24 hours, to something much lower, like 1 hour.  You can check the current value, shown in minutes, using the command below.

stsadm.exe -o getproperty -propertyname token-timeout

In my case the value had already been reduced to 1 hour, so I decided to set it back to the default of 24 hours using the command below.  This resolved the error.

stsadm.exe -o setproperty -propertyname token-timeout -propertyvalue 1440

Windows cannot install required files error 0x80070570

I recently encountered an issue where a server wouldn’t install any Windows Updates, whether directly from Microsoft, or if downloaded from the Windows Update Catalog.  The error message received was “Windows cannot install required files error 0x80070570” and this is how I resolved the error.

From an administrative command prompt I ran the System File Checker using sfc /scannow.  This stopped with an error at 36%.  The log for sfc is in the %windir%\Logs\CBS folder.

Next, I used the Deployment Image Servicing and Management (DISM) tool to perform a health scan of the Windows system files.  Again, I ran this from an administrative command prompt, Dism /Online /Cleanup-Image /ScanHealth, but it showed no errors.

WUError-1

Despite the fact no corruption was detected, I ran the following command to fix corruption, Dism /Online /Cleanup-Image /RestoreHealth.  In my case, rather than using Windows Update to retrieve fresh files, I used the /Source switch to point to a copy of the SXS folder from a Windows installation ISO.

WUError-2

Having fixed the corruption Windows Updates now installed.

SQL Server backup performance on Azure Premium Storage

In this post I’ll show the results of some basics tests I carried out to identify the optimum number of SQL Server backup files to use when backing up to an Azure Premium Storage P30 disk with two different VM sizes.

The tests were carried out using SQL Server 2016 Standard SP2 on Windows Server 2016.  All drives were formatted with a 64K allocation size and backup compression was enabled.   The database was 123GB with 9GB free space.

The virtual machine configuration was:

  • DS14-8 v2 8 cores and 112GB RAM
  • 1x P30 with read caching enabled for database data files
  • 1x P30 with caching disabled for database log files
  • 1x P30 with caching disabled for backup files

The DS14-8 v2 supports 51,200 IOPs and 768MBps throughput, so has capacity to support the 15,000 IOPs and 600MBps that the 3x P30 disks can generate.  The results show that only two backup files are required for close to peak performance.

1 backup file: 152.648 MB/sec

2 backup files: 192.484 MB/sec

4 backup files: 198.223 MB/sec

8 backup files: 194.735 MB/sec

I then resized the VM to a DS13 v2, to see if the reduced VM storage capability would impact the backup performance.  The DS13 v2 has 8 cores, 56GB RAM and supports 25,600 uncached IOPs and 384MBps throughput.  The IOPs capability is sufficient to support all three P30 disks, but the throughput could be limiting.  However, the results show very little difference.

1 backup file: 150.416 MB/sec

2 backup files: 192.969 MB/sec

4 backup files: 194.692 MB/sec

8 backup files: 194.561 MB/sec

Monitor Azure events with Logic Apps – follow up

Following on from my earlier post “Monitor Azure events with Logic Apps” I’ll now show how to add additional control within the Logic App to only alert on events related to virtual machine changes and how to surface additional event information in the email.

Firstly edit the Logic App in the designer and click the + between the event and email steps and choose Add an action.

EG0

From the Choose an action screen select Control.

EG1

Select Condition.

EG2

Click into the “Choose a value” box, select Expression and enter triggerBody()?[‘data’][‘operationName’] and click ok.

EG3

In the next box select “is equal to” and in the value box enter Microsoft.Compute/virtualMachine.

EG4

Drag the “Send an email” action into the “If true” box.

EG5

These changes ensure the Logic App will only send an email for operations related to virtual machines.

If you would like to include additional event information in the email beyond those in the Dynamic content selection it’s easy to achieve.  From the Logic App page click on runs history.  Click on “When a resource event occurs” and you can see the JSON on the event. Click “Show raw outputs” to see the full JSON generated by the event.

EG56

You can include values from the JSON in the email using the expression box.  Go back to the “Send an email” action and click on the email body.  In the Expression box enter the path to the JSON value, e.g. triggerBody()?[‘data’][‘operationName’] to include the operation name or triggerBody()[‘data’][‘claims’][‘http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name’%5D to include the name of the user that carried out the action.

EG7