');--success-svg-icon:url('data:image/svg+xml;utf8,');--info-svg-icon:url('data:image/svg+xml;utf8,');--chevron-svg-icon:url('data:image/svg+xml;utf8,');--checkbox-svg-icon:url('data:image/svg+xml;utf8,');--radiobutton-svg-icon:url('data:image/svg+xml;utf8,');--show-password-svg-icon:url('data:image/svg+xml;utf8,');--hide-password-svg-icon:url('data:image/svg+xml;utf8,');--error-svg-icon-color:var(--red-500);--success-svg-icon-color:var(--emerald-700);--info-svg-icon-color:var(--sky-500);--chevron-svg-icon-color:var(--grey-900);--checkbox-svg-icon-color:var(--grey-900);--radiobutton-svg-icon-color:var(--grey-900);--show-password-svg-icon-color:var(--grey-900);--hide-password-svg-icon-color:var(--grey-900);--bde-woo-notices__icon-size:16px;--bde-woo-notices__padding:24px;--bde-woo-notices__border-radius:4px;--bde-woo-notices__error-background:var(--red-50);--bde-woo-notices__error-text:var(--red-500);--bde-woo-notices__error-link-text:var(--red-500);--bde-woo-notices__error-link-text-hover:var(--red-500);--bde-woo-notices__info-background:var(--sky-100);--bde-woo-notices__info-text:var(--sky-500);--bde-woo-notices__info-link-text:var(--sky-500);--bde-woo-notices__info-link-text-hover:var(--sky-500);--bde-woo-notices__success-background:var(--emerald-100);--bde-woo-notices__success-text:var(--emerald-700);--bde-woo-notices__success-link-text:var(--emerald-700);--bde-woo-notices__success-link-text-hover:var(--emerald-700);--bde-woo-sale-badge__border-radius:2px;--bde-woo-sale-badge__padding:4px 8px;--bde-woo-ratings__star-color:var(--yellow-500);--bde-woo-ratings__star-size:18px;--bde-woo-ratings__filled-star-svg:url('data:image/svg+xml;utf8,');--bde-woo-ratings__empty-star-svg:url('data:image/svg+xml;utf8,');--bde-woo-product-images__border-radius:4px;--bde-woo-product-images__border-width:0px;--bde-woo-product-images__border-color:var(--grey-300);--bde-woo-wrappers__background-color:var(--white);--bde-woo-wrappers__border-radius:4px;--bde-woo-wrappers__border-color:var(--grey-300);--bde-woo-wrappers__border-width:1px;--bde-woo-wrappers__shadow:rgba(0,0,0,.05) 0 1px 3px,rgba(0,0,0,.05) 0 1px 2px;--bde-woo-tables__header-color:var(--grey-100);--bde-woo-tables__background-color:var(--white);--bde-woo-tables__border-radius:4px;--bde-woo-tables__border-width:1px;--bde-woo-widgets__chip-background-color:transparent;--bde-woo-widgets__chip-background-color-hover:var(--indigo-50);--bde-woo-widgets__chip-text-color:inherit;--bde-woo-widgets__handle-border-color:var(--grey-300);--bde-woo-widgets__handle-background-color:var(--white);--bde-woo-widgets__handle-background-color-hover:var(--white);--bde-woo-widgets__handle-shadow:rgba(0,0,0,.05) 0 1px 3px,rgba(0,0,0,.05) 0 1px 2px;--bde-woo-widgets__handle-shadow-hover:var(--grey-300) 0 0 4px;--bde-woo-widgets__remove-item-from-cart-color:var(--grey-450);--bde-woo-select2__active-item-background-color:var(--indigo-50);--bde-woo-gallery__zoom-icon-shadow:rgba(0,0,0,.05) 0 1px 3px,rgba(0,0,0,.05) 0 1px 2px;--bde-woo-payment-box-background-color:var(--grey-100);--bde-woo-payment-box-border-color:var(--grey-200);--bde-woo-payment-box-border-width:1px;--bde-woo-quicklook-button-icon:url(/wp-content/uploads/breakdance/css/icons/eye.svg);--bde-woo-quicklook-button-background-color:none;--bde-woo-quicklook-button-icon-size:20px;--bde-woo-quicklook-button-backdrop-color:rgba(0,0,0,.15);--bde-woo-quicklook-button-backdrop-opacity:.7;--bde-woo-quicklook-modal-background-color:var(--white);--bde-woo-quicklook-overlay-color:rgba(0,0,0,.7);--bde-woo-quicklook-close-button-size:2em;--bde-woo-quicklook-arrow-size:1em;--bde-woo-quicklook-arrow-color:var(--white);--bde-woo-swatch-space-between-options:10px;--bde-woo-swatch-padding:7px;--bde-woo-swatch-color-width:30px;--bde-woo-swatch-color-height:30px;--bde-woo-swatch-border:1px solid var(--grey-200);--bde-woo-swatch-border-color-hover:var(--grey-400);--bde-woo-swatch-shadow:none;--bde-woo-swatch-shadow-hover:none;--bde-woo-swatch-background:var(--white);--bde-woo-swatch-background-hover:var(--grey-50);--bde-woo-swatch-color-padding:2px;--bde-woo-swatch-tooltip-color:var(--white);--bde-woo-swatch-tooltip-background:var(--grey-900);--bde-woo-swatch-tooltip-padding:7px;--bde-woo-swatch-space-after-label:8px;--bde-woo-responsive__stack:row;--bde-links-color:#d61f2c;--bde-links-color-hover:#d61f2c;--bde-button-primary-background-color:var(--bde-brand-primary-color);--bde-button-primary-background-color-hover:var(--bde-brand-primary-color-hover);--bde-button-secondary-border-color:var(--bde-brand-primary-color);--bde-button-secondary-border-color-hover:var(--bde-brand-primary-color);--bde-button-secondary-text-color:var(--bde-brand-primary-color);--bde-button-secondary-background-color-hover:var(--bde-brand-primary-color);--bde-button-text-text-color:var(--bde-links-color);--bde-button-text-text-color-hover:var(--bde-links-color-hover);--bde-button-font-size:var(--bde-body-font-size);--bde-form-font-size:var(--bde-body-font-size);--bde-form-input-border-top-left-radius:var(--bde-form-input-border-radius);--bde-form-input-border-top-right-radius:var(--bde-form-input-border-radius);--bde-form-input-border-bottom-left-radius:var(--bde-form-input-border-radius);--bde-form-input-border-bottom-right-radius:var(--bde-form-input-border-radius);--bde-form-input-border-top:var(--bde-form-input-border-width) solid var(--bde-form-input-border-color);--bde-form-input-border-right:var(--bde-form-input-border-width) solid var(--bde-form-input-border-color);--bde-form-input-border-bottom:var(--bde-form-input-border-width) solid var(--bde-form-input-border-color);--bde-form-input-border-left:var(--bde-form-input-border-width) solid var(--bde-form-input-border-color);--bde-form-label-color:var(--bde-headings-color);--bde-form-input-focused-border-color:var(--bde-brand-primary-color);--bde-form-input-focused-shadow:var(--bde-brand-primary-color) 0 0 2px;--bde-form-checkbox-selected-color:var(--bde-brand-primary-color);--bde-z-index-lightbox:1100;--bde-z-index-popup:1050;--bde-z-index-modal:1000;--bde-z-index-modal-backdrop:calc(var(--bde-z-index-modal) - 1);--bde-z-index-high:300;--bde-z-index-medium:200;--bde-z-index-low:100;--bde-z-index-sticky:10;--bde-z-index-minicart:var(--bde-z-index-modal);--bde-z-index-minicart-backdrop:var(--bde-z-index-modal-backdrop);--bde-z-index-menu-dropdown:var(--bde-z-index-modal);--bde-z-index-menu-mobile:var(--bde-z-index-modal);--bde-z-index-menu-backdrop:var(--bde-z-index-modal-backdrop);--bde-z-index-search-fullscreen:var(--bde-z-index-modal);--bde-z-index-back-to-top:var(--bde-z-index-high);--bde-z-index-scroll-progress:var(--bde-z-index-high);--bde-z-index-header-sticky:var(--bde-z-index-medium);--bde-z-index-header-overlay:calc(var(--bde-z-index-header-sticky) - 1);--bde-z-index-social-share-buttons:var(--bde-z-index-low);--bde-woo-base-transition:all ease var(--bde-transition-duration);--bde-woo-base-text-color:var(--bde-body-text-color);--bde-woo-base-headings-color:var(--bde-headings-color);--bde-woo-base-primary-color:var(--bde-brand-primary-color);--bde-woo-base-primary-color-hover:var(--bde-brand-primary-color-hover);--bde-woo-base-extra-small-gaps:calc(var(--bde-woo-base-small-gaps)/var(--bde-woo-base-ratio));--bde-woo-base-small-gaps:calc(var(--bde-woo-base-space)/var(--bde-woo-base-ratio));--bde-woo-base-standard-gaps:var(--bde-woo-base-space);--bde-woo-base-medium-gaps:calc(var(--bde-woo-base-space)*var(--bde-woo-base-ratio));--bde-woo-base-big-gaps:calc(var(--bde-woo-base-medium-gaps)*var(--bde-woo-base-ratio));--bde-woo-base-large-gaps:calc(var(--bde-woo-base-big-gaps)*var(--bde-woo-base-ratio));--bde-woo-base-extra-large-gaps:calc(var(--bde-woo-base-large-gaps)*var(--bde-woo-base-ratio));--bde-woo-typography-ratio:var(--bde-font-size-ratio);--bde-woo-typography__size-small:calc(var(--bde-woo-typography__size-standard)/var(--bde-woo-typography-ratio));--bde-woo-typography__size-small-font-family:var(--bde-body-font-family);--bde-woo-typography__size-standard:var(--bde-body-font-size);--bde-woo-typography__size-standard-font-family:var(--bde-body-font-family);--bde-woo-typography__size-medium:calc(var(--bde-woo-typography__size-standard)*var(--bde-woo-typography-ratio));--bde-woo-typography__size-medium-font-family:var(--bde-heading-font-family);--bde-woo-typography__size-large:calc(var(--bde-woo-typography__size-medium)*var(--bde-woo-typography-ratio));--bde-woo-typography__size-large-font-family:var(--bde-heading-font-family);--bde-woo-typography__size-extra-large:calc(var(--bde-woo-typography__size-large)*var(--bde-woo-typography-ratio));--bde-woo-typography__size-extra-large-font-family:var(--bde-heading-font-family);--bde-woo-buttons-and-links__text-link-color:#d61f2c;--bde-woo-buttons-and-links__text-link-color-hover:#d61f2c;--bde-woo-buttons-and-links__nav-link-color:var(--bde-woo-base-text-color);--bde-woo-buttons-and-links__nav-link-color-hover:var(--bde-woo-base-text-on-primary-color);--bde-woo-buttons-and-links__nav-link-color-active:var(--bde-woo-base-primary-color);--bde-woo-forms__spacing-after-label:var(--bde-form-after-label);--bde-woo-forms__spacing-between-fields:var(--bde-form-gap);--bde-woo-forms__spacing-between-columns:var(--bde-woo-base-extra-large-gaps);--bde-woo-forms__labels-color:var(--bde-form-label-color);--bde-woo-forms__inputs-background-color:var(--bde-form-input-background-color);--bde-woo-forms__inputs-text-color:var(--bde-form-text-color);--bde-woo-forms__inputs-placeholder-color:var(--bde-form-input-placeholder-color);--bde-woo-forms__inputs-border-color:var(--bde-form-input-border-color);--bde-woo-forms__inputs-border-width:var(--bde-form-input-border-width);--bde-woo-forms__inputs-border-radius:var(--bde-form-input-border-top-left-radius) var(--bde-form-input-border-top-right-radius) var(--bde-form-input-border-bottom-right-radius) var(--bde-form-input-border-bottom-left-radius);--bde-woo-forms__inputs-background-color-focused:var(--bde-form-input-focused-background-color);--bde-woo-forms__inputs-border-color-focused:var(--bde-form-input-focused-border-color);--bde-woo-forms__inputs-shadow-focused:var(--bde-form-input-focused-shadow);--bde-woo-forms__inputs-shadow:var(--bde-form-input-input-shadow);--bde-woo-forms__inputs-select2-hover-item:var(--bde-woo-base-primary-color);--bde-woo-forms__labels-required-color:var(--bde-form-label-required-color);--bde-woo-forms__labels-required-size:var(--bde-form-label-required-size);--bde-woo-forms__labels-required-nudge-x:var(--bde-form-label-required-nudge-x);--bde-woo-forms__labels-required-nudge-y:var(--bde-form-label-required-nudge-y);--bde-woo-tables__border-color:var(--bde-woo-base-border-color);--bde-woo-sale-badge__background-color:var(--bde-woo-base-primary-color);--bde-woo-sale-badge__text-color:var(--bde-woo-base-text-on-primary-color);--bde-woo-sale-badge__font-weight:var(--bde-woo-typography-font-weight-heavy);--bde-woo-sale-badge__font-size:var(--bde-woo-typography__size-standard);--bde-woo-widgets__chip-text-color-hover:var(--bde-woo-buttons-and-links__text-link-color);--bde-woo-widgets__handle-border-color-hover:var(--bde-woo-base-primary-color);--bde-woo-notices__padding-left:calc(var(--bde-woo-notices__padding) + var(--bde-woo-notices__icon-size) + (var(--bde-woo-notices__icon-size)/2));--bde-woo-quicklook-button-text-color:var(--bde-button-primary-text-color);--bde-woo-quicklook-button-background-color-hover:var(--bde-woo-quicklook-button-background-color);--bde-woo-quicklook-button-icon-spacing:var(--bde-woo-base-standard-gaps);--bde-woo-quicklook-close-button-color:var(--bde-woo-base-headings-color);--bde-woo-quicklook-arrow-color-hover:var(--bde-woo-quicklook-arrow-color);--bde-woo-quicklook-arrow-background-color:var(--bde-brand-primary-color);--bde-woo-quicklook-arrow-background-color-hover:var(--bde-woo-quicklook-arrow-background-color);--bde-woo-swatch-space-between-items:var(--bde-woo-base-medium-gaps);--bde-woo-swatch-background-selected:var(--bde-woo-swatch-background-hover);--bde-woo-swatch-border-color-selected:var(--bde-woo-swatch-border-color-hover);--bde-woo-swatch-shadow-selected:var(--bde-woo-swatch-shadow-hover);--bde-woo-swatch-color-background:var(--bde-woo-swatch-background)}.breakdance-icon-atom{display:flex;font-size:40px}.breakdance-icon-atom>svg:not(.breakdance-icon-atom-svg-gradient){font-size:inherit;width:1em;height:1em;stroke-width:0;stroke:var(--bde-brand-primary-color);fill:var(--bde-brand-primary-color);transform:var(--eeiTransform)}.breakdance *,.breakdance :after,.breakdance :before{box-sizing:border-box}.breakdance img{max-width:100%;height:auto}.breakdance figure{margin-left:0;margin-right:0}.breakdance{background-color:var(--bde-background-color);color:var(--bde-body-text-color);font-family:var(--bde-body-font-family);font-size:var(--bde-body-font-size);color:#0c163d;font-family:"DM Sans",sans-serif;font-size:18px;font-weight:400;line-height:28px}.breakdance h1,.breakdance h2,.breakdance h3{font-family:var(--bde-heading-font-family);font-size:var(--bde-h1-font-size);color:var(--bde-brand-primary-color);font-size:clamp(1.2rem,1.25rem + 2vw,2.35rem);font-weight:600;line-height:1.3}.breakdance h2,.breakdance h3{color:var(--bde-headings-color);font-size:var(--bde-h2-font-size);font-size:clamp(.5rem,.9rem + .8vw,1.875rem)}.breakdance a{color:var(--bde-links-color);text-decoration:none;text-decoration-thickness:2px;text-decoration-style:none;text-decoration-color:var(--bde-links-color)}.breakdance a:hover{color:var(--bde-links-color-hover);text-decoration-thickness:2px;text-decoration-line:underline;text-decoration-style:solid}.breakdance h3{font-size:var(--bde-h3-font-size);font-size:clamp(.5rem,.9rem + .8vw,1.25rem)}:root{--bde-section-vertical-padding:60px}@media (max-width:479px){.breakdance h2{font-size:24px}.breakdance h3{font-weight:600}}.breakdance .bde-section-42631-108 .section-container{display:flex;flex-direction:column;align-items:center;gap:24px;text-align:center}.breakdance .bde-section-42631-108{background-color:var(--bde-palette-color-1-1169274a-8713-4b28-8077-f42d9212950f)}.breakdance .bde-heading-42631-175{margin-bottom:0}.breakdance .bde-section-42631-150 .section-container{display:flex;flex-direction:column;align-items:flex-start;justify-content:center;text-align:left}.breakdance .bde-section-42631-150{background-color:var(--bde-palette-color-1-1169274a-8713-4b28-8077-f42d9212950f)}.breakdance .bde-columns-42631-159{--columnCount:2}.breakdance .bde-columns-42631-159>div{padding:16px}@media (max-width:1119px){.breakdance .bde-columns-42631-159{--columnCount:2}.breakdance .bde-columns-42631-159.bde-columns{flex-direction:column;flex-wrap:nowrap}.breakdance .bde-columns-42631-159.bde-columns>.bde-column{width:100%}.breakdance .bde-columns-42631-159>div:nth-child(1){order:10}.breakdance .bde-columns-42631-159>div:nth-child(2){order:9}}@media (max-width:1023px){.breakdance .bde-columns-42631-159{--columnCount:2}}@media (max-width:767px){.breakdance .bde-columns-42631-159{--columnCount:2}}@media (max-width:479px){.breakdance .bde-columns-42631-159{--columnCount:2}}.breakdance .bde-column-42631-161{--column-width:75%;display:flex;flex-direction:column;gap:24px;background-color:var(--bde-background-color)}.breakdance .bde-text-42631-163{margin-top:0}.breakdance .bde-div-42631-194{padding:12px;border-bottom:2px solid var(--bde-brand-primary-color);display:flex;flex-direction:row;flex-wrap:nowrap;justify-content:center;align-items:flex-start;gap:16px}.breakdance .bde-image-42631-195{width:400px}.breakdance .bde-image-42631-195 .breakdance-image-object{width:100%;height:auto;filter:grayscale()}.breakdance .bde-image-42631-195 .breakdance-image__caption{text-align:center;color:var(--bde-background-color);font-weight:600;background:var(--bde-links-color);padding:4px}.breakdance .bde-column-42631-162{--column-width:25%;display:flex;flex-direction:column;align-items:flex-start;gap:32px;text-align:left;background-color:var(--bde-background-color)}.breakdance .bde-social-share-buttons-42631-166{justify-content:flex-start;--shbtn-icon-size:20px;--shbtn-padding:0px}.breakdance .bde-social-share-buttons-42631-166 .bde-social-share-button-icon{padding-inline-end:var(--shbtn-padding)}.breakdance .bde-social-share-buttons-42631-166 .bde-social-share-button{background-color:var(--bde-background-color);border-color:var(--bde-background-color)}.breakdance .bde-social-share-buttons-42631-166 .bde-social-share-button-icon{color:var(--bde-headings-color)}@media (max-width:1119px){.breakdance .bde-social-share-buttons-42631-166{justify-content:flex-start}}@media (max-width:1023px){.breakdance .bde-social-share-buttons-42631-166{justify-content:flex-start}}@media (max-width:767px){.breakdance .bde-social-share-buttons-42631-166{justify-content:flex-start}}@media (max-width:479px){.breakdance .bde-social-share-buttons-42631-166{justify-content:flex-start}}.breakdance .bde-post-meta-42631-172{background-color:#ffeef0;color:var(--bde-links-color);font-weight:600;letter-spacing:.5px;padding:6px 12px;border-radius:6px}.breakdance .bde-table-of-contents-42631-201 .bde-table-of-contents__header,.breakdance .bde-table-of-contents-42631-201 .bde-table-of-contents__wrapper{background-color:#f7f7f7}.breakdance .bde-table-of-contents-42631-201 .bde-table-of-contents__list{padding-top:0}.breakdance .bde-table-of-contents-42631-201 .toc-list .is-active-li::marker{font-weight:600;color:var(--bde-links-color)}.breakdance .bde-section-42631-182 .section-container{align-items:center;text-align:center;justify-content:center}.breakdance .bde-section-42631-182{background-color:#f3f3f3}@media (max-width:1119px){.breakdance .bde-section-42631-182 .section-container{padding-bottom:60px;padding-top:60px}}@media (max-width:1023px){.breakdance .bde-section-42631-182 .section-container{padding-bottom:60px;padding-top:40px}}@media (max-width:767px){.breakdance .bde-section-42631-182 .section-container{padding-bottom:60px;padding-top:20px}}@media (max-width:479px){.breakdance .bde-section-42631-182 .section-container{padding-bottom:80px;padding-top:20px}}.breakdance .bde-heading-42631-183{color:#000;margin-bottom:48px}.breakdance .bde-post-list-42631-184>.ee-posts{--bde-posts-gap:16px;--bde-posts-per-row:3}.breakdance .bde-post-list-42631-184 .ee-post{background-color:var(--bde-background-color);border-radius:12px;box-shadow:0 0 4px 0#00000025;padding:20px;flex-direction:column;gap:16px;align-items:flex-start}.breakdance .bde-post-list-42631-184 .ee-post-wrap{align-items:flex-start}.breakdance .bde-post-list-42631-184 .ee-post-meta-item:not(:last-child):after{content:"•";color:#00386b;margin:0 8px}.breakdance .bde-post-list-42631-184 .ee-post-title{margin-bottom:16px;text-align:left}.breakdance .bde-post-list-42631-184 .ee-post-title a{font-size:20px}.breakdance .bde-post-list-42631-184 .ee-post-image img{border-radius:6px}.breakdance .bde-post-list-42631-184 .ee-post-meta{margin-bottom:12px;justify-content:flex-start}.breakdance .bde-post-list-42631-184 .ee-post-meta-item{color:var(--bde-brand-primary-color);font-size:18px;font-weight:600}@media (max-width:1119px){.breakdance .bde-post-list-42631-184>.ee-posts{--bde-posts-per-row:3}}@media (max-width:1023px){.breakdance .bde-post-list-42631-184>.ee-posts{--bde-posts-per-row:2}}@media (max-width:767px){.breakdance .bde-post-list-42631-184>.ee-posts{--bde-posts-per-row:1}.breakdance .bde-post-list-42631-184{flex-direction:column;align-items:unset}}@media (max-width:479px){.breakdance .bde-post-list-42631-184>.ee-posts{--bde-posts-per-row:1}}.breakdance .bde-section-41845-383 .section-container{display:flex;flex-direction:row;flex-wrap:nowrap;justify-content:space-between;gap:20px;padding-bottom:10px;padding-top:10px}.breakdance .bde-section-41845-383{background-color:var(--bde-headings-color);box-shadow:0 12px 25px 0#00000025}@media (max-width:767px){.breakdance .bde-section-41845-383 .section-container{justify-content:space-between;align-items:center}}@media (max-width:479px){.breakdance .bde-section-41845-383 .section-container{justify-content:space-around;gap:6px;padding-left:0;padding-right:0}}.breakdance .bde-div-41845-384{display:flex;flex-direction:row;flex-wrap:nowrap;justify-content:center;align-items:center;gap:32px}@media (max-width:767px){.breakdance .bde-div-41845-384{justify-content:center;align-items:flex-start;gap:4px}}@media (max-width:479px){.breakdance .bde-div-41845-384{justify-content:center;align-items:flex-start;gap:6px}}.breakdance .bde-icon-list-41845-416 li .bde-icon-list__icon{font-size:24px;color:var(--bde-background-color)}.breakdance .bde-icon-list-41845-416 li .bde-icon-list__text{color:var(--bde-background-color);font-weight:600}.breakdance .bde-icon-list-41845-414 li:hover .bde-icon-list__text,.breakdance .bde-icon-list-41845-415 li:hover .bde-icon-list__text,.breakdance .bde-icon-list-41845-416 li:hover .bde-icon-list__text,.breakdance .bde-icon-list-41845-417 li:hover .bde-icon-list__text{color:#fff}@media (max-width:767px){.breakdance .bde-icon-list-41845-416 li .bde-icon-list__icon{font-size:24px}.breakdance .bde-icon-list-41845-416 li .bde-icon-list__text{font-size:0}}@media (max-width:479px){.breakdance .bde-icon-list-41845-416 li .bde-icon-list__text{font-weight:600}}.breakdance .bde-icon-list-41845-417 li .bde-icon-list__icon{font-size:24px;color:var(--bde-background-color)}.breakdance .bde-icon-list-41845-417 li .bde-icon-list__text{color:var(--bde-background-color);font-weight:600}@media (max-width:767px){.breakdance .bde-icon-list-41845-417 li .bde-icon-list__icon{font-size:24px}.breakdance .bde-icon-list-41845-417 li .bde-icon-list__text{font-size:0}}@media (max-width:479px){.breakdance .bde-icon-list-41845-417 li .bde-icon-list__text{font-weight:600}}.breakdance .bde-div-41845-391{display:flex;flex-direction:row;flex-wrap:nowrap;justify-content:center;align-items:center;gap:16px}@media (max-width:767px){.breakdance .bde-div-41845-391{align-items:flex-start;gap:12px}}@media (max-width:479px){.breakdance .bde-div-41845-391{gap:12px}}.breakdance .bde-icon-list-41845-414 li .bde-icon-list__icon{font-size:24px;color:var(--bde-background-color)}.breakdance .bde-icon-list-41845-414 li .bde-icon-list__text{color:var(--bde-background-color);font-weight:600}@media (max-width:767px){.breakdance .bde-icon-list-41845-414 li .bde-icon-list__icon{font-size:24px}.breakdance .bde-icon-list-41845-414 li .bde-icon-list__text{font-size:12px}}@media (max-width:479px){.breakdance .bde-icon-list-41845-414 li .bde-icon-list__text{font-weight:600}}.breakdance .bde-icon-list-41845-415 li .bde-icon-list__icon{font-size:24px;color:var(--bde-background-color)}.breakdance .bde-icon-list-41845-415 li .bde-icon-list__text{color:var(--bde-background-color);font-weight:600}@media (max-width:767px){.breakdance .bde-icon-list-41845-415 li .bde-icon-list__icon{font-size:24px}.breakdance .bde-icon-list-41845-415 li .bde-icon-list__text{font-size:12px}}@media (max-width:479px){.breakdance .bde-icon-list-41845-415 li .bde-icon-list__text{font-weight:600}}.breakdance .bde-header-builder-41845-398{background:var(--bde-background-color);box-shadow:0 1px 1px 0#00000025}.breakdance .bde-header-builder-41845-398 .bde-header-builder__container{padding-bottom:6px;padding-top:6px}@media (max-width:1023px){.breakdance .bde-header-builder-41845-398 .bde-header-builder__container{padding-left:20px;padding-right:20px}}.breakdance .bde-image-41845-399{width:230px}.breakdance .bde-image-41845-399 .breakdance-image-object{width:100%;height:auto}@media (max-width:1119px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-background:#fff;--menu-hide-on-mobile:none;--toggle-layer-color:#000;--mobile-offcanvas-x:100%;--mobile-topbar-close-button-color:var(--bde-links-color);--link-border:none}}@media (min-width:1120px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-duration:.5s;--menu-hide-on-desktop:none;--links-gap:12px;--link-color:var(--bde-headings-color);--link-color-hover:var(--bde-brand-primary-color);--link-padding-top:12px;--link-padding-right:12px;--link-padding-bottom:12px;--link-padding-left:12px;--link-effect-color:var(--bde-brand-primary-color);--link-effect-color-hover:var(--bde-brand-primary-color);--link-effect-thickness:2px;--link-arrow-color-hover:var(--bde-brand-primary-color);--dropdown-custom-offset-y:6.2px}.breakdance .bde-menu-41845-400 .breakdance-menu-link{color:var(--bde-headings-color);text-transform:capitalize}.breakdance .bde-menu-41845-400 .breakdance-menu-item:hover .breakdance-menu-link{color:var(--bde-brand-primary-color)}}@media (max-width:1119px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-mobile:none;--mobile-topbar-logo-width:200px;--link-color:#000}.breakdance .bde-menu-41845-400 .breakdance-menu-link{color:#000;font-size:18px}.breakdance .bde-menu-41845-400 .breakdance-menu{--links-gap:0;--dropdown-link-icon-display:none;--dropdown-link-description-display:none;--dropdown-links-gap:var(--gap-none)}.breakdance .bde-menu-41845-400 .breakdance-menu-list{align-items:stretch;background-color:var(--menu-background);display:none;flex-direction:column;left:var(--mobile-offset-x);max-height:var(--menu-max-height);overflow-y:auto;position:absolute;top:var(--mobile-offset-y,100%);width:calc(100vw - var(--menu-scrollbar-width,0px));z-index:var(--bde-z-index-menu-mobile)}.breakdance .bde-menu-41845-400 .breakdance-menu-list>:last-child{border-bottom:var(--link-border)}.breakdance .bde-menu-41845-400 .breakdance-menu .breakdance-menu-link{border-top:var(--link-border);outline-offset:-1px;padding:var(--mobile-link-padding-top) var(--mobile-link-padding-right) var(--mobile-link-padding-bottom) var(--mobile-link-padding-left)}.breakdance .bde-menu-41845-400 .breakdance-menu-link-arrow{border-top:var(--link-border);bottom:0;position:absolute;right:0;top:0}.breakdance .bde-menu-41845-400 .breakdance-dropdown-floater{transform:none;visibility:visible}.breakdance .bde-menu-41845-400 .breakdance-dropdown-body{background-color:var(--link-background-color);border-radius:0;border-top:var(--link-border);box-shadow:none;flex-direction:column}.breakdance .bde-menu-41845-400 .breakdance-dropdown--custom .breakdance-dropdown-floater{width:auto}.breakdance .bde-menu-41845-400 .breakdance-dropdown-section{--dropdown-section-padding:0;--dropdown-section-gap:var(--gap-none)}.breakdance .bde-menu-41845-400 .breakdance-dropdown-columns{--dropdown-columns-stack:column;--dropdown-columns-gap:var(--gap-none)}.breakdance .bde-menu-41845-400 .breakdance-dropdown-column+.breakdance-dropdown-column{border-top:var(--link-border)}.breakdance .bde-menu-41845-400 .breakdance-dropdown-column,.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible{gap:var(--gap-none)}.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible .breakdance-dropdown-item:first-child,.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible:not(:first-child){border-top:var(--link-border)}.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible .breakdance-dropdown-column__title{cursor:pointer;outline-offset:-1px}.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible .breakdance-dropdown-links:not(.is-visible):not(.is-collapsing){height:0;overflow:hidden;pointer-events:none;visibility:hidden}.breakdance .bde-menu-41845-400 .breakdance-dropdown-column__title{font-weight:500;justify-content:var(--link-alignment);padding:var(--mobile-link-level-2-padding-top) var(--mobile-link-level-2-padding-right) var(--mobile-link-level-2-padding-bottom) var(--mobile-link-level-2-padding-left);white-space:normal}.breakdance .bde-menu-41845-400 .breakdance-menu--collapse .breakdance-dropdown-column--collapsible .breakdance-dropdown-column__title:after{display:block}.breakdance .bde-menu-41845-400 .breakdance-dropdown-item+.breakdance-dropdown-item{border-top:var(--link-border)}.breakdance .bde-menu-41845-400 .breakdance-menu .breakdance-dropdown-link{justify-content:var(--link-alignment);outline-offset:-1px;padding:var(--mobile-link-level-3-padding-top) var(--mobile-link-level-3-padding-right) var(--mobile-link-level-3-padding-bottom) var(--mobile-link-level-3-padding-left);white-space:normal}.breakdance .bde-menu-41845-400 .breakdance-menu-topbar{align-items:center;display:flex}.breakdance .bde-menu-41845-400 .breakdance-menu-toggle{display:flex}@media (min-width:1120px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-desktop:none}}}@media (max-width:1023px){@media (max-width:1119px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-mobile:none;--mobile-topbar-logo-width:200px;--link-color:#000}.breakdance .bde-menu-41845-400 .breakdance-menu-link{color:#000;font-size:18px;font-weight:500}}@media (min-width:1120px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-desktop:none}}}@media (max-width:767px){@media (max-width:1119px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-mobile:none;--mobile-topbar-logo-width:200px}}@media (min-width:1120px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-desktop:none}}}@media (max-width:479px){@media (max-width:1119px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-mobile:none;--mobile-topbar-logo-width:230px}}@media (min-width:1120px){.breakdance .bde-menu-41845-400 .breakdance-menu{--menu-hide-on-desktop:none}}}@media (min-width:1120px){.breakdance .bde-menu-dropdown-41845-401 .breakdance-dropdown{--dropdown-section-padding:12px}.breakdance .bde-menu-dropdown-41845-401 .breakdance-dropdown-body{border-radius:12px;box-shadow:0 0 12px 0#00000025}.breakdance .bde-menu-dropdown-41845-401 .breakdance-dropdown .breakdance-dropdown-column__title,.breakdance .bde-menu-dropdown-41845-402 .breakdance-dropdown .breakdance-dropdown-column__title{font-weight:600}.breakdance .bde-menu-dropdown-41845-401 .breakdance-dropdown{--dropdown-links-gap:8px}.breakdance .bde-menu-dropdown-41845-402 .breakdance-dropdown{--dropdown-section-padding:12px}.breakdance .bde-menu-dropdown-41845-402 .breakdance-dropdown-body{border-radius:12px;box-shadow:0 0 12px 0#00000025}.breakdance .bde-menu-dropdown-41845-402 .breakdance-dropdown{--dropdown-links-gap:8px}.breakdance .bde-menu-dropdown-41845-403 .breakdance-dropdown{--dropdown-custom-width:200px;--dropdown-sections-stack:row;--dropdown-section-padding:16px}.breakdance .bde-menu-dropdown-41845-403 .breakdance-dropdown-body{border-radius:12px;box-shadow:0 0 12px 0#00000025}.breakdance .bde-menu-dropdown-41845-404 .breakdance-dropdown{--dropdown-custom-width:200px;--dropdown-sections-stack:row;--dropdown-section-padding:16px}.breakdance .bde-menu-dropdown-41845-404 .breakdance-dropdown-body{border-radius:12px;box-shadow:0 0 12px 0#00000025}.breakdance .bde-menu-dropdown-41845-405 .breakdance-dropdown{--dropdown-custom-width:200px;--dropdown-sections-stack:row;--dropdown-section-padding:16px}.breakdance .bde-menu-dropdown-41845-405 .breakdance-dropdown-body{border-radius:12px;box-shadow:0 0 12px 0#00000025}}.breakdance .bde-section-41847-100 .section-container{display:flex;flex-direction:column;padding-bottom:60px;padding-top:60px}.breakdance .bde-section-41847-100{background-color:var(--bde-brand-primary-color)}@media (max-width:1023px){.breakdance .bde-section-41847-100 .section-container{gap:16px}}.breakdance .bde-image-41847-104{width:230px}.breakdance .bde-image-41847-104 .breakdance-image-object{width:100%;height:auto}.breakdance .bde-grid-41847-101{--bde-grid-items-per-row:3}.breakdance .bde-grid>*{width:auto!important;height:auto!important}@media (max-width:1119px){.breakdance .bde-grid-41847-101{--bde-grid-items-per-row:3}}@media (max-width:1023px){.breakdance .bde-grid-41847-101{--bde-grid-items-per-row:2}}@media (max-width:767px){.breakdance .bde-grid-41847-101{--bde-grid-items-per-row:1}}.breakdance .bde-div-41847-102{display:flex;flex-direction:column;align-items:flex-start;gap:12px;text-align:left}@media (max-width:1023px){.breakdance .bde-div-41847-102{padding-left:16px}}@media (max-width:767px){.breakdance .bde-div-41847-102{padding-left:16px}}@media (max-width:479px){.breakdance .bde-div-41847-102{padding-left:16px}}.breakdance .bde-text-41847-105{color:var(--bde-background-color);font-size:16px}@media (max-width:767px){.breakdance .bde-text-41847-105{font-size:15px}}.breakdance .bde-social-icons-41847-106{flex-direction:row}.breakdance .bde-social-icons-41847-106 .bde-social-icons__icon-wrapper{border-radius:100%;background-color:var(--bde-background-color);padding:12px;opacity:1}.breakdance .bde-social-icons-41847-106 .bde-social-icons__icon-wrapper:hover{background-color:#efefef;transform:scale(1.1);opacity:.8}.breakdance .bde-social-icons-41847-106 .bde-social-icons__icon-wrapper svg{width:16px;height:16px;fill:var(--bde-brand-primary-color)}.breakdance .bde-div-41847-107{display:flex;flex-direction:column;gap:6px}.breakdance .bde-text-41847-137{color:var(--bde-background-color);font-weight:600}.breakdance .bde-wp-menu-41847-135{width:100%}.breakdance .bde-wp-menu-41847-135 .breakdance-menu{--link-alignment:flex-start;--menu-hide-on-desktop:none;--link-color:var(--bde-background-color);--link-padding-top:10px;--link-padding-bottom:10px}.breakdance .bde-wp-menu-41847-135 .breakdance-menu-link{color:var(--bde-background-color)}@media (max-width:1119px){.breakdance .bde-wp-menu-41847-135 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:1023px){.breakdance .bde-wp-menu-41847-135 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:767px){.breakdance .bde-wp-menu-41847-135 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:479px){.breakdance .bde-wp-menu-41847-135 .breakdance-menu{--menu-hide-on-desktop:none}}.breakdance .bde-div-41847-119{display:flex;flex-direction:column;gap:6px}.breakdance .bde-text-41847-138{color:var(--bde-background-color);font-weight:600}.breakdance .bde-div-41847-131,.breakdance .bde-wp-menu-41847-136{width:100%}.breakdance .bde-wp-menu-41847-136 .breakdance-menu{--link-alignment:flex-start;--menu-hide-on-desktop:none;--link-color:var(--bde-background-color);--link-padding-top:10px;--link-padding-bottom:10px}.breakdance .bde-wp-menu-41847-136 .breakdance-menu-link{color:var(--bde-background-color)}@media (max-width:1119px){.breakdance .bde-wp-menu-41847-136 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:1023px){.breakdance .bde-wp-menu-41847-136 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:767px){.breakdance .bde-wp-menu-41847-136 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:479px){.breakdance .bde-wp-menu-41847-136 .breakdance-menu{--menu-hide-on-desktop:none}}.breakdance .bde-div-41847-131{display:flex;flex-direction:row;flex-wrap:nowrap;justify-content:space-between;align-items:center}@media (max-width:767px){.breakdance .bde-div-41847-131{flex-direction:column;flex-wrap:nowrap;justify-content:flex-start}}.breakdance .bde-text-41847-132{color:var(--bde-background-color)}@media (max-width:767px){.breakdance .bde-text-41847-132{font-size:15px}}.breakdance .bde-wp-menu-41847-133 .breakdance-menu{--menu-hide-on-desktop:none;--link-color:var(--bde-background-color)}.breakdance .bde-wp-menu-41847-133 .breakdance-menu-link{color:var(--bde-background-color)}@media (max-width:1119px){.breakdance .bde-wp-menu-41847-133 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:1023px){.breakdance .bde-wp-menu-41847-133 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:767px){.breakdance .bde-wp-menu-41847-133 .breakdance-menu{--menu-hide-on-desktop:none}}@media (max-width:479px){.breakdance .bde-wp-menu-41847-133 .breakdance-menu{--menu-hide-on-desktop:none}}
Performance Guidance for SQL Server in Microsoft Azure Virtual Machines
SQL Server Technical Article
Summary: Developers and IT professionals should be fully knowledgeable about how to optimize the performance of SQL Server workloads running in Microsoft Azure Infrastructure Services and in more traditional on-premises environments. This technical article discusses the key factors to consider when evaluating performance and planning a migration to SQL Server in Azure Virtual Machines. It also provides certain best practices and techniques for performance tuning and troubleshooting when using SQL Server in Microsoft Azure Infrastructure Services.
The goal of this technical article is to provide guidance to developers and IT professionals on how to optimize the performance of Microsoft SQL Server workloads running in an Azure Virtual Machine environment. The article first describes the key concepts in Azure that have a direct impact on the performance of SQL Server workloads. It then introduces the key factors to take into account when evaluating performance and when you plan a migration to Azure platform. Next, it introduces additional considerations for performance troubleshooting in Azure Infrastructure Services. The article uses results from specific test scenarios to provide guidance and best practices for optimizing the performance of SQL Server workloads running in Azure virtual machine (VM).
The online documentation for SQL Server in Azure Virtual Machines covers getting started, migration, deployment, high availability, security, connectivity, backup and restore, creating a new virtual machine using one of our pre-built SQL Server platform images in the Image Gallery, and other topics. For more information, see SQL Server in Azure Virtual Machines.
The following is a quick check list that you can follow:
• Use minimum Standard Tier A2 for SQL Server VMs.
• Keep the storage account and SQL Server VM in the same region.
• Disable Azure geo-replication on the storage account.
• Avoid using operating system or temporary disks for database storage or logging.
• Avoid using Azure data disk caching options (caching policy = None).
• Stripe multiple Azure data disks to get increased IO throughput.
• Format with documented allocation sizes.
• Separate data and log file I/O paths to obtain dedicated IOPs for data and log.
• Enable database page compression.
• Enable instant file initialization for data files.
• Limit or disable autogrow on the database.
• Disable autoshrink on the database.
• Move all databases to data disks, including system databases.
• Move SQL Server error log and trace file directories to data disks.
• Apply SQL Server performance fixes.
• Setup default locations.
• Enable locked pages.
• Backup directly to blob storage.
For more information, please follow the guidelines provided in the following sub sections.
This section provides an overview of the Azure Infrastructure Services including some key considerations that can have a direct impact on the performance of your SQL Server workloads.
Azure Infrastructure Services lets you access scalable, on-demand infrastructure using Virtual Machines and Virtual Networks. You can use Azure to store data, to create virtual machines for development and test, and to build web applications or run other applications. Before you start reading this article, you should understand the fundamental concepts of Azure, such as cloud services, virtual machines, virtual networks, storage accounts, and so on. For more information, see Azure Documentation.
Azure virtual machines are available in different sizes and resource options in terms of number of CPU cores, memory capacity, maximum disk space, bandwidth, and so on. For the latest information about the supported virtual machine sizes and resource options, see Virtual Machine and Cloud Service Sizes for Azure. We recommend that you review all these virtual machine size options and choose the ones best suited for your SQL Server workloads.
The network traffic includes all traffic between client applications and SQL Server in Azure VM and, any other communication that involves a SQL Server process, or other processes in a virtual machine, such as ETL packages or backup and restore operations. We recommend that you choose a larger virtual machine size with more network bandwidth.
Important note: Network communications generated by accessing data disks and the operating system disk attached to the virtual machine are not considered part of the bandwidth limits.
Azure Virtual Machines provide three types of disks:
Important note: Proper configuration of your Azure disks (operating system and data) is one of the most important areas to focus on when optimizing the performance of your SQL Server workloads. Both operating system disks and data disks are stored as VHD files in the individual page blobs hosted in the Azure Storage Blob Service. Each blob has its own capacity limits based on the intrinsic storage architecture. For more information, see the following blog posts:
Each data disk can be a maximum of 1 TB in size. Depending upon your virtual machine size, you can attach up to 16 data disks to your virtual machine. I/Os per second (IOPS) and bandwidth are the most important factors to consider when determining how many data disks you need.
Storage capacity planning is similar to the traditional ‘spindle’ count in an on-premises storage design. In this article, we discuss various approaches you can use to spread I/O operations across one or more data disks, and discuss how to use SQL Server performance features, such as compression, to maximize the performance of your SQL Server workloads.
In Azure virtual machines, data of persisted drives is cached locally to the host machine, thus bringing the data closer to the virtual machine. Azure disks (operating system and data) use a two-tier cache:
Cache settings help reduce the number of transactions against Azure Storage and can reduce disk I/O latency in some scenarios.
Azure disks support three different cache settings: Read Only, Read Write, and None (disabled). There is no configuration available to control the size of the cache.
The following table demonstrates the supported disk cache modes:
Disk type | Read Only | Read Write | None (disabled) |
Operating system disk | Supported | Default mode | Not supported |
Data disk | Supported | Supported | Default mode |
Note that temporary disks are not included in this table because they are not Azure disks, but are implemented using local attached storage.
Important notes:
For specific recommendations and best practices for cache settings and performance behaviors, see the Azure virtual machine disks and cache settings section in this article.
Before you migrate your SQL Server workloads to Azure Infrastructure Services, consider the hardware, design, and architectural differences between your on-premises environment and Azure Infrastructure Services. Knowing these differences will help you identify which workloads are appropriate for this environment. The Microsoft Assessment and Planning (MAP) Toolkit can help you assess your current IT infrastructure for various platform migrations, including Azure Virtual Machines. You should also consider adopting an approach that is well suited to measure the performance of your SQL Server workloads in Azure Infrastructure Services.
Storage optimization for SQL Server transactional and analytical workloads is a very important task that requires careful planning and analysis. There is already a tremendous amount of information that explains how to deal with I/O subsystems with different performance characteristics in a traditional on-premises environment, such as spindles, host bus adapters (HBAs), disk controllers, and so on. For more information, see Analyzing I/O Characteristics and Sizing Storage Systems for SQL Server Database Applications.
Azure disks are implemented as a service, so they do not offer the same range of complex configuration options available in traditional on-premises I/O subsystems. This has both benefits and costs. For instance, Azure disks offer built-in local redundancy and optional geo-redundancy for disaster recovery through the use of replicas. To achieve the same level of redundancy in on-premises deployments, you would need to set up multiple disk arrays in multiple locations and a synchronization mechanism, such as, a storage area network (SAN) replication. On the other hand, the Azure disk performance is not as predictable as on-premises disk I/O subsystem due to several factors:
For a detailed discussion on different storage configurations, see the Best practices and recommendations for optimizing SQL Server performance in Azure VMs section.
Although Azure Storage introduces most of the differences between on-premises and Azure deployments, other system components, such as CPU and memory, need to be considered as well when you evaluate performance. In Azure, the only configurable option for CPU is the number of CPU cores assigned to a virtual machine deployment, currently from one shared (A0) to sixteen dedicated in A9 in the Standard Tier, which is subject to increase or change in the future. These CPU cores may not be the same as the SKUs customers can find in expensive on-premises servers. This can lead to significant performance differences in CPU-bound workload and needs to be taken into account during testing and baselining.
For memory, the current offering spans from 768 MB in A0 to 112GB in an A9 Compute Intensive Instance in the Standard Tier. Again, when you compare the performance characteristics of an existing on-premises application, you need to consider the right virtual machine size and its other options to avoid any performance impact due to inadequate memory sizing for buffer pools or other internal memory structures.
Network latency in Azure Infrastructure Services can be higher than that of a traditional on-premises environment, due to several reasons, such as virtualization, security, load balancers, and so on. This means that reducing network round trips between application layers in a cloud solution has a strong positive impact on the performance when compared to on-premises solutions.
For “chatty” applications, where communications between application layers and components are frequent, we recommend that you consolidate multiple application layers on the same virtual machine. This reduces the number of tiers and the amount of communications that your application needs resulting in better performance.
When this is not possible or even counterproductive, consider the following recommendations to optimize network communication:
To achieve optimal results in any complex storage environment, you need to follow a systematic, data- driven methodology. This section summarizes the approach that we use internally within Microsoft to develop best practices on how to configure Azure Infrastructure Services and SQL Server for various types of workloads.
To perform I/O benchmarking tests on different Azure disk configurations, we use the Microsoft SQLIO Disk Subsystem Benchmark Tool and analyze the various performance metrics as described in the Analyzing I/O Characteristics and Sizing Storage Systems for SQL Server Database Applications article through the various performance counters and dynamic management views (DMVs). The primary goal of these tests is to determine the I/O capacity of various configurations. Then, we compare these configurations with different SQL Server workloads (for example, OLTP, point lookup queries, analytical queries with aggregations) and I/O patterns. Our goal is to verify which configuration is most appropriate for each type of workload. The Best practices and recommendations for optimizing SQL Server performance in Azure VMs section contains our findings and recommendations based on these results.
Note that precise I/O performance depends on a number of factors including data center capacity and network utilization. So, you should consider these generalized findings as subject to change. We hope that they will be useful for capacity planning as you consider which SQL Server workloads you want to migrate to Azure environment.
We recommend that you also run your own validation tests. You can see a detailed example of reusable scripts in the Appendix.
Many of the same techniques used to optimize SQL Server performance in your on-premises environment can be used to tune your SQL Server workloads in Azure Infrastructure Services. Having said that, running your database workload in a hosted multi-tenant cloud service like Azure Infrastructure Services is fundamentally different and if you want to be successful you will need to consider some new best practices. This section provides new best practices and recommendations for optimizing SQL Server performance in Azure Infrastructure Services.
In general, smaller VM sizes are best suited for lightweight development and test systems or applications. For production and intensive workload deployments, bigger virtual machine sizes (such as A3 or high memory instances) are often a better choice because they can provide more capacity in terms of virtual cores, memory and data disks.
For SQL Server production workloads, we recommend that you use minimum Standard Tier A2 VM sizes or bigger instances. Starting with May 2014, new VM sizes (A8 and A9) have been introduced sporting faster Intel Xeon processor and increased memory sizes. Based on various performance tests, these VMs provide important benefits in terms of CPU performance, IO throughput and bandwidth. If you plan to run very high SQL Server workloads in Azure Virtual Machines, we recommend that you consider these new VM sizes.
Azure Virtual Machines provide three types of disks: operating system (OS) disk, temporary disk, and data disks. For a description of each disk type, see section Azure Infrastructure services fundamentals in this article.
When placing your data and log files you should consider disk cache settings in addition to size limits. For a description of cache settings, see section Azure Infrastructure services fundamentals in this article.
While “Read Write” cache (default setting) for the operating system disk helps improve the overall operating system performance, boot times and reducing the read latency for the IO patterns that OS usually generates, we recommend that you do not use OS disk for hosting system and user database files. Instead, we recommend that you use data disks. When the workload demands a high rate of random I/Os (such as a SQL Server OLTP workload) and throughput is important to you, the general guideline is to keep the cache set to the default value of “None” (disabled). Because Azure storage is capable of more IOPS than a direct attached storage disk, this setting causes the physical host local disks to be bypassed, therefore providing the highest I/O rate.
Unlike Azure disks (operating system and data disks) which are essentially VHDs stored as page blobs in Azure Storage, the temporary disk (labeled as D:) is not persistent and is not implemented using Azure Storage. It is reserved by the operating system for the page file and its performance is not guaranteed to be predictable. Any data stored on it may be lost after your virtual machine is restarted or resized. Hence, we do not recommend the D: drive for storing any user or system database files, including tempdb.
This section discusses the best practices and recommendations on data disk performance options based on testing done by Microsoft. You should be familiar with how SQL Server I/O operations work in order to interpret the test results reported in this section. For more information, see Pages and Extents Architecture.
It is important to note that the results we provide in this section were achieved without SQL Server High Availability and Disaster Recovery Solutions enabled (such as, AlwaysOn Availability Groups, database mirroring or log shipping). We recommend that you deploy one of these features to maintain multiple redundant copies of your databases across at least two virtual machines in an availability set in order to be covered by the Azure Cloud Services, Virtual Machines, and Virtual Network Service Level Agreement. Enabling any of these features affects performance, so you should consider incorporating one of them in your own performance testing to get more accurate results.
As a general rule, we recommend that you attach maximum number of disks allowed by the VM size (such as, 16 data disks for an A7 VM) for throughput sensitive applications. While latency may not necessarily improve by adding more data disks when your workload is within the maximum IOPS limit, the additional IOPS and bandwidth that you get from the attached additional disks can help to avoid reaching the single disk 500 IOPS limit. Note that this might trigger throttling events that might increase disk response times and disk latency.
In our performance tests, we’ve executed several SQL Server I/O measurements to understand data disk response characteristics with respect to the typical I/O patterns generated by SQL Server based on different kind of workloads. The results for a single disk configuration on an A7 VM instance are summarized here:
Random I/O (8 KB Pages) | Sequential I/O (64 KB Extents) | |||||
Reads | Writes | Reads | Writes | |||
IOPS | 500 | 500 | 500 | 300 | ||
Bandwidth | 4 MB/s | 4 MB/s | 30 MB/s | 20 MB/s |
Note: Because Azure Infrastructure Services is a multi-tenant environment, performance results may vary. You should consider these results as an indication of what you can achieve, but not a guarantee. We suggest you repeat these tests and measurements based on your specific workload.
If your workload exceeds or is close to the I/O performance numbers mentioned in the previous section, we recommend that you add multiple disks (depending on your virtual machine size) and stripe multiple disks in volumes. This configuration gives you the ability to create volumes with specific throughput and bandwidth, based on your data and log performance needs by combining multiple data disks together.
After you create a virtual machine in Azure, you can attach a data disk to it using either the Azure Management Portal or the Add-AzureDataDisk Azure PowerShell cmdlet. Both techniques allow you to select an existing data disk from a storage account, or create a new blank data disk.
If you choose to create a new blank data disk at the Management Portal, you can choose the storage account that your virtual machine was created in but not a different storage account.
To place your existing data disk (.vhd file) into a specific storage account, you need to use the Azure PowerShell cmdlets. The following example demonstrates how to update a virtual machine using the Get-AzureVM and the Add-AzureDataDisk cmdlets. The Get-AzureVM cmdlet retrieves information on a specific virtual machine. The Add-AzureDataDisk cmdlet creates a new data disk with specified size and label in a previously created Storage Account.
Get-AzureVM “CloudServiceName” -Name “VMNAme” | Add-AzureDataDisk -CreateNew -DiskSizeInGB 100 -MediaLocation ` “https://<storageaccount>.blob.core.windows.net/vmdisk/Disk1.vhd” -DiskLabel “disk1” -LUN 1 | Update-AzureVM
To create a new storage account, use the New-AzureStorageAccount cmdlet as follows:
New-AzureStorageAccount -StorageAccountName “StorageAccountX” -Label “StorageAccountX” -Location “North Central US”
For more information about Azure PowerShell cmdlets, see Azure PowerShell on MSDN and Azure command line tools.
For Azure VMs running on Windows Server 2008 R2 and previous releases, the only striping technology available is striped volumes for dynamic disks. You can use this option to stripe multiple data disks into volumes that provide more throughput and bandwidth than what a single disk can provide.
Starting with Windows Server 2012, Storage Pools are introduced and operating system software RAID capabilities are deprecated. Storage Pools enable you to virtualize storage by grouping industry-standard disks into “pools”, and then create virtual disks called Storage Spaces from the available capacity in the storage pools. You can then configure these virtual disks to provide striping capabilities across all disks in the pool, combining good performance characteristics. In addition, it enables you to add and remove disk space based on your needs.
During our tests, after adding a number of data disks (4, 8 and 16) as shown in the previous section, we created a new storage pool by using the following Windows PowerShell command:
New-StoragePool –FriendlyName StoragePool1 –StorageSubsystemFriendlyName “Storage Spaces*” –PhysicalDisks (Get-PhysicalDisk –CanPool $True)
Next, we created a virtual disk on top of the new storage pool and specified resiliency setting and virtual disk size.
$disks = Get-StoragePool –FriendlyName StoragePool1 -IsPrimordial $false | Get-PhysicalDisk
New-VirtualDisk –FriendlyName VirtualDisk1 -ResiliencySettingName Simple –NumberOfColumns $disks.Count –UseMaximumSize –Interleave 256KB
Important Note: For performance, it is very important that the –NumberOfColumns parameter is set to the number of disks utilized to create the underlying Storage Pool. Otherwise, IO requests cannot be evenly distributed across all data disks in the pool and you will get suboptimal performance.
The –Interleave parameter enables you to specify the number of bytes written in each underlying data disk in a virtual disk. We recommend that you use 256 KB for all workloads.
Lastly, we created and formatted the volume to make it usable to the operating system and applications by using the following Windows PowerShell commands:
Get-VirtualDisk –FriendlyName VirtualDisk1 | Get-Disk | Initialize-Disk –Passthru | New-Partition –AssignDriveLetter –UseMaximumSize | Format-Volume –AllocationUnitSize 64K
Once the volume created, it is possible to dynamically increase the disk capacity by attaching new data disks. To achieve optimal capacity utilization, consider the number of columns your storage spaces have and add disks in multiples of that number. See Windows Server Storage spaces Frequently Asked Questions for more information.
Using Storage Pools instead of traditional Windows operating system striping in dynamic disks brings several advantages in terms of performance and manageability. We recommend that you use Storage Pools for disk striping in Azure Virtual Machines.
During our internal testing, we have implemented the following scenarios with different number of disks as well as disk volume configurations. We tested the following scenarios with configurations of 4, 8 and 16 data disks respectively, and we observed increased IOPS for each data disk added as expected:
It’s important to notice that using multiple data disks provides performance benefits but it creates more management overhead. In addition, partial unavailability of one of the striped disks can result in unavailability of a database. Therefore, for such configurations, we recommend that you consider enhancing the availability of your databases using high availability and disaster recovery capabilities of SQL Server as described in High Availability and Disaster Recovery for SQL Server in Azure Virtual Machines.
The following tables summarize the results of tests that we performed using multiple data disks configurations at Microsoft.
Aggregated throughput and bandwidth across 4 data disks
Random I/O (8 KB Pages) | Sequential I/O (64 KB Extents) | |||||
Reads | Writes | Reads | Writes | |||
IOPS | 2000 | 2000 | 1600 | 1200 | ||
Bandwidth | 16 MB/s | 16 MB/s | 100 MB/s | 75 MB/s |
Aggregated throughput and bandwidth across 8 data disks
Random I/O (8 KB Pages) | Sequential I/O (64 KB Extents) | |||||
Reads | Writes | Reads | Writes | |||
IOPS | 4000 | 4000 | 2400 | 2400 | ||
Bandwidth | 30 MB/s | 30 MB/s | 150 MB/s | 150 MB/s |
Aggregated throughput and bandwidth across 16 data disks
Random I/O (8 KB Pages) | Sequential I/O (64 KB Extents) | |||||
Reads | Writes | Reads | Writes | |||
IOPS | 8000 | 8000 | 2400 | 4000 | ||
Bandwidth | 60 MB/s | 60 MB/s | 150 MB/s | 250 MB/s |
Note: Because Azure Infrastructure Services is a shared, multi-tenant environment, performance results may vary. You should consider these results as an indication of what you can achieve, but not a guarantee. We recommend that you repeat these tests and measurements based on your specific workload.
By using the newly introduced Intel-based A8 and A9 VM sizes, we repeated our IO performance tests and noticed a significant increase in bandwidth and throughput for larger sequential IO requests. If you use Intel-based A8 and A9 VM sizes, you can get a performance increase for 64 KB (and bigger) read and write operations. If your workload is IO intensive, these new VM sizes (A8 and A9) can help in achieving more linear scalability compare to smaller VM sizes, but always within the 500 IOPs per disk boundaries. For more information, see About the A8 and A9 Compute Intensive Instances.
Based on our tests, we have made the following observations about the Azure Virtual Machine environment:
Depending on how you configure your storage, you should place and the data and log files for user and system databases accordingly to achieve your performance goals. This section provides guidance on how you should place database files when using SQL Server in Azure Virtual Machines:
This option can give you precise file placement by optimizing available IO performance. You can find the graphical representation of this configuration below:
You can still create single disk volumes and leverage SQL Server files and filegroups placement for your databases. While this can still offer some benefits in terms of flexible storage layout organization, it introduces additional complexity and also limits single file (data or log) IO performance to a value that a single Azure data disk can provide such as 500 IOPs and 60 MB/sec.
Although Azure data disks have different behaviors than traditional rotating spindles (,in which competing random and sequential operations on the same disks can impact performance), we still recommend that you keep data and log files in different paths to achieve dedicated IOPs and bandwidth for them.
To help understand your IO requirements and performance while running your SQL Server workloads on Azure Virtual Machines, you need to analyze the following three tools and combine the results carefully:
By leveraging all these information, you can understand:
As mentioned in section Azure virtual machine disks and cache settings, we recommend that you place tempDB on data disks instead of the temporary disk (D:). Following are the three primary reasons for this recommendation based on our internal testing with SQL Server test workloads.
We strongly recommend that you perform your own workload testing before implementing a desired SQL Server file layout strategy.
With Azure disks, we have observed a “warm-up effect” that can result in a reduced rate of throughput and bandwidth for a short period of time. In situations where a data disk is not accessed for a period of time (approximately 20 minutes), adaptive partitioning and load balancing mechanisms kick in. If the disk is accessed while these algorithms are active, you may notice some degradation in throughput and bandwidth for a short period of time (approximately 10 minutes), after which they return to their normal levels. This warm-up effect happens because of the adaptive partitioning and load balancing mechanism of Azure, which dynamically adjusts to workload changes in a multi-tenant storage environment. You may observe similar effects in other widely known cloud storage systems as well. For more information, see Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency.
This warm-up effect is unlikely to be noticed for systems that are in continuous use. But we recommend you consider it during performance testing or when accessing systems that have been inactive for a while.
To simplify management and reduce potential risks of consistency in case of failures, we recommend that you leave all the data disks attached to a single virtual machine in the same storage account. Storage accounts are implemented as a recovery unit in case of failures. So, keeping all the disks in the same account makes the recovery operations simple. There is no performance improvement if you store data disks attached to a single VM in multiple storage accounts. If you have multiple VMs, we recommend that you consider the storage account limits for throughput and bandwidth during capacity planning. In addition, distribute VMs and their data disks to multiple storage accounts if the aggregated throughput or bandwidth is higher than what a single storage account can provide. For information on storage account limits, see Azure Storage Scalability and Performance Targets. For information on max IOPS per disk, see Virtual Machine and Cloud Service Sizes for Azure.
NTFS volumes use a default cluster size of 4 KB. Based on our performance tests, we recommend changing the default cluster size to 64 KB during volume creation for both single disk and multiple disks (storage spaces) volumes.
Some I/O intensive workloads can gain performance benefits through data compression. Compressed tables and indexes means more data stored in fewer pages, and hence require reading fewer pages from disk, which in turn can improve the performance of workloads that are I/O intensive.
For a data warehouse workload running on SQL Server in Azure VM, we found significant improvement in query performance by using page compression on tables and indexes, as shown in Figure 1.
Figure 1: Query Performance with Data Compression
Figure 1 compares performance of one query with no compression (NONE) and page compression (PAGE). As illustrated, the logical and physical reads are significantly reduced with page compression, and so is the elapsed time. As expected, CPU time of the query does go up with page compression, because SQL Server needs to decompress the data while returning results to the query. Your results will vary, depending upon your workload.
For an OLTP workload, we observed significant improvements in throughput (as measured by business transactions per second) by using page compression on selected tables and indexes that were involved in the I/O intensive workload. Figure 2 compares the throughput and CPU usage for the OLTP workload with and without page compression.
Figure 2: OLTP Throughput and CPU Usage with Data Compression
Note that you may see different results when you test your workloads in Azure Virtual Machine environment. But we recommend that you test data compression techniques for I/O intensive workloads and then decide which tables and indexes to compress. For more information, see Data Compression: Strategy, Capacity Planning and Best Practices.
For databases of any significant size, enabling instant file initialization can improve the performance of some operations involving database files, such as creating a database or restoring a database, adding files to a database or extending the size of an existing file, autogrow, and so on. For information, see How and Why to Enable Instant File Initialization.
To take advantage of instant file initialization, you grant the SQL Server (MSSQLSERVER) service account with SE_MANAGE_VOLUME_NAME and add it to the Perform Volume Maintenance Tasks security policy. If you are using a SQL Server platform image for Azure, the default service account (NT Service\MSSQLSERVER) isn’t added to the Perform Volume Maintenance Tasks security policy. In other words, instant file initialization is not enabled in a SQL Server Azure platform image.
After adding the SQL Server service account to the Perform Volume Maintenance Tasks security policy, restart the SQL Server service.
The following figure illustrates observed test results for creating and restoring a 100 GB database with and without instant file initialization.
Figure 3: Performance Impact of Instant File Initialization
For more information, see Database File Initialization.
Many of the best practices when running SQL Server on premises are still relevant in Azure Virtual Machines, including:
This section provides an overview of how you troubleshoot SQL Server when running on a Azure virtual machine.
Performance analysis in SQL Server is well documented. The Troubleshooting Performance Problems in SQL Server 2008 article provides comprehensive information on this subject. You can also find other advanced performance-related blogs and articles on topics such as latch and spinlock analysis at SQLCAT.com. Here, let’s summarize the main performance factors:
Although the SQL Server binaries running in a traditional on-premises and Microsoft Azure virtual machine environments are identical, there are some infrastructure differences that affect the way that applications perform in SQL Server in Azure Virtual Machines compared to an on-premises dedicated enterprise server. An analogy that can be used to compare the experience of running SQL Server in Azure virtual machine is an on-premises virtualized SQL Server environment with dedicated virtualized resources (such as processor and memory) but no Hyper-V over-commit of memory or processors. In addition, the I/O performance varies in Microsoft Azure storage as it is a managed shared disk environment and performance at a point in time depends on other tenants’ activity and overall system load; this can happen in shared on-premises storage environments as well.
All the traditional SQL Server performance factors described in the previous section are still applicable when running SQL Server in Azure virtual machine environment. The following table summarizes most common resource bottleneck issues and provides a list of actions that you can take to resolve them.
Issue | Key performance indicators (KPIs) to monitor | Actions to consider |
SQL Server CPU at or near 80% | % Processor Time (_Total)SOS_SCHEDULER_YIELD waits |
|
Near I/O capacity limits orI/O latency increases | Average disk reads per secondAverage disk writes per second Disk reads per second Disk writes per second io_virtual_file_stats PAGEIOLATCH waits SQL Server: Buffer Manager\Page Life Expectancy |
Note: This can apply to any user created or tempdb databases.
|
Memory resource pressure | Memory: Available BytesMemory: Pages per second SQL Server: Buffer Manager\Page Life Expectancy Process: Working Set (for SQL Server) RESOURCE_SEMAPHORE waits |
|
As described in the table, you can resolve performance issues by following different approaches and actions. In traditional on-premises environment, you might prefer adding or purchasing more hardware resources to alleviate performance e problems. In Azure environment, the resources that are available per machine are smaller in number and less powerful than the typical on-premises enterprise servers. For example, while adding a data disk may increase disk throughput by 25 percent; tuning query workloads might reduce the overall I/O requirement by 90 percent in some cases. Therefore we recommend that you always follow a systematic approach that involves analyzing, tuning and redesigning to achieve better performance results.
Optimizing your application for Azure Virtual Machine environment will provide valuable cost benefits because you can achieve a higher density per unit of compute. Unlike the traditional on-premises environment, Azure allows you reduce the number and size virtual machines immediately to reduce the operational costs. In addition, you can dynamically re-balance the size of machines based on seasonal usage peaks.
It is important that you develop a systematic monitoring and troubleshooting methodology to effectively run your SQL Server in Azure Virtual Machine environment.
This section explains the approaches that can be used to identify and isolate performance issues while running SQL Server in Azure Virtual Machine environment:
Other common profiles depend on the predominant wait type and include: SOS_SCHEDULER_YIELD, which indicates that the operation is waiting on the CPU to become free; long PAGEIOLATCH waits, which indicate that SQL Server is generating I/O requests faster than the I/O system can process them; and RESOURCE_SEMAPHORE, which indicates memory pressure on memory. See the Snapshot wait stats script provided in the Appendix. It demonstrates how to automate the calculation of waits during a time interval. This is useful for identifying the dominant underlying wait in SQL Server when symptoms appear with your workload.
Logical disk counter | Typical storage term | Suggested actions in Microsoft Azure virtual machine environment |
Disk reads / secondDisk writes / second | IOPS | Measure the number of I/O’s per second.Consider adding more data disks in line with your IOPS requirements. |
Average disk sec / readAverage disk sec / write | Latency | Measure disk latency.Note: Numbers might vary; look at averages over time. |
Average disk bytes / readAverage disk bytes / write | Block size | Measure the size of I/O’s being issued.Note: Larger I/O’s tend to have higher latency, such as those associated with BACKUP/RESTORE. |
Average / current disk queue length | Outstanding or waiting IOPS | Provides insight into the applications I/O pattern. |
Disk read bytes/secDisk write bytes/sec | Throughput or aggregate throughput | Measure of total disk throughput.Note: Ideally, larger block scans should be able to heavily utilize connection bandwidth (for example, your throughput can be higher with a smaller number of larger IOPS). |
The scripts and guidance covered in this section can be used to facilitate the information given in this article:
You can run the following SQLIO scripts in Azure Virtual Machine to generate some of the most common I/O patterns that SQL Server utilizes and measure related performance results on different storage configurations.
The following SQLIO test scripts demonstrate testing random 8K reads/writes (a typical OLTP I/O pattern), sequential writes for log files, and large sequential reads and writes for table scans and OLAP workloads.
The script uses the following options:
Copy and save the following script in a file called exectests.bat.
::Test random 8K reads/writes |
sqlio -kW -s300 -frandom -o32 -b8 -LS -Fparam.txt |
sqlio -kR -s300 -frandom -o32 -b8 -LS -Fparam.txt |
::Test random 32K reads/writes (for example, SQL Server Analysis Services I/O pattern)sqlio -kW -s300 -frandom -o32 -b32 -LS -Fparamnew.txt sqlio -kR -s300 -frandom -o32 -b32 -LS -Fparamnew.txt |
::Test small sequential writes |
sqlio -kW -s180 -fsequential -o1 -b4 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o1 -b8 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o1 -b16 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o1 -b32 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o1 -b64 -LS -Fparam.txt |
::Test large sequential reads/writes |
sqlio -kR -s180 -fsequential -o8 –b8 -LS -Fparam.txt |
sqlio -kR -s180 -fsequential -o8 -b64 -LS -Fparam.txt |
sqlio -kR -s180 -fsequential -o8 -b128 -LS -Fparam.txt |
sqlio -kR -s180 -fsequential -o8 -b256 -LS -Fparam.txt |
sqlio -kR -s180 -fsequential -o8 –b512 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o8 –b8 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o8 -b64 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o8 -b128 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o8 -b256 -LS -Fparam.txt |
sqlio -kW -s180 -fsequential -o8 –b512 -LS -Fparam.txt |
The following is a copy of the param.txt configuration file that we referenced in our test scripts. Basically, the test scripts run a series of tests against the drive or drives specified in the param.txt file. This file should reside in the same directory as SQLIO.exe. The options on each line of the param.txt file are as follows, where 0x0 is a mask value:
PathToTestFile NumberofThreadsPerTestFile 0x0 TestFileSizeinMegaBytes
This param.txt file skips operating system and temporary disks (C: and D:) and tests a 16 data disks volume (F:) (using 1 thread per disk), with a file size of 50 GB.
#c:\testfile.dat 2 0x0 100 |
#d:\testfile.dat 1 0x0 1000 |
f:\testfile.dat 16 0x0 50000 |
Open a Command Prompt window, then run the test batch file as follows.
exectests.bat > results.txt
This operation captures all test results in a text file that can be processed manually or automatically to extract relevant disk performance figures later.
We recommend that you define key performance indicators (KPIs) for all of the components in your application scenario. Typically, you can use performance counters to define your KPIs.
This section lists KPIs that we’ve used to monitor application tier performance and user characteristics while running SQL Server in Azure. Note that the KPIs listed in this section are meant to be guidance only. You should define your KPIs based on the characteristics of your application’s components.
Typical SQL Server KPIs:
If your workload is likely to be I/O bound, you should also add the Logical Disk Performance Monitor counters referenced earlier. If you use additional SQL Server features, such as AlwaysOn, it is recommended that you add the appropriate performance monitor counters.
Typical web application tier KPIs:
Typical user/test characteristics KPI:
First, identify five to eight KPIs for each application component, such as SQL Server, application tier, and a similar number for the test characteristics KPI’s. The KPIs you choose should be a subset of overall performance counter collection. Note that these KPIs represent an overall performance of the application during different time intervals. In a production environment, you may notice KPIs return different information during the day. We recommend that you calculate and analyze KPIs regularly by using tools, such as Excel. For example, you should be able to make assertions such as “When we scale from one application server to two application servers, the CPU utilization on my SQL Server increases by 2.5x”. In this example, you could continue investigating further using detailed Performance Monitor logs and DMV stats to understand why SQL Server CPU utilization has increased by 2.5x and understand whether this is a normal characteristic of our workload or there is an issue we need to investigate.
You can automate the collection of the raw performance logs and the calculation of the KPIs by using the Logman and Log Parser tools.
/* Snapshot the current wait stats and store them so that they can be compared over a time period
Return the statistics between this point in time and the last collection point in time.
*/
use tempdb
go
if exists (select * from sys.objects where name = ‘snap_waits’)
drop procedure snap_waits
go
create procedure snap_waits
as
declare @current_snap_time datetime
declare @previous_snap_time datetime
set @current_snap_time = GETDATE()
if not exists(select name from tempdb.sys.sysobjects where name like ‘wait_stats%’)
create table wait_stats
(
wait_type varchar(128)
,waiting_tasks_count bigint
,wait_time_ms bigint
,avg_wait_time_ms int
,max_wait_time_ms bigint
,signal_wait_time_ms bigint
,avg_signal_wait_time int
,snap_time datetime
)
insert into wait_stats (
wait_type
,waiting_tasks_count
,wait_time_ms
,max_wait_time_ms
,signal_wait_time_ms
,snap_time
)
select
wait_type
,waiting_tasks_count
,wait_time_ms
,max_wait_time_ms
,signal_wait_time_ms
,@current_snap_time
from sys.dm_os_wait_stats
–get the previous collection point
select top 1 @previous_snap_time = snap_time from wait_stats
where snap_time < (select max(snap_time) from wait_stats)
order by snap_time desc
–get delta in the wait stats
select top 10
s.wait_type
, (e.wait_time_ms – s.wait_time_ms)/((e.waiting_tasks_count – s.waiting_tasks_count)) as [avg_wait_time_ms]
,(e.signal_wait_time_ms – s.signal_wait_time_ms)/((e.waiting_tasks_count – s.waiting_tasks_count)) as [avg_signal_time_ms]
, (e.waiting_tasks_count – s.waiting_tasks_count) as [waiting_tasks_count]
, (e.wait_time_ms – s.wait_time_ms) as [wait_time_ms]
, (e.max_wait_time_ms) as [max_wait_time_ms]
, (e.signal_wait_time_ms – s.signal_wait_time_ms) as [signal_wait_time_ms]
, s.snap_time as [start_time]
, e.snap_time as [end_time]
, DATEDIFF(ss, s.snap_time, e.snap_time) as [seconds_in_sample]
from wait_stats e
inner join (
select * from wait_stats
where snap_time = @previous_snap_time
) s on (s.wait_type = e.wait_type)
where
e.snap_time = @current_snap_time
and s.snap_time = @previous_snap_time
and e.wait_time_ms – s.wait_time_ms > 0
and e.waiting_tasks_count – s.waiting_tasks_count > 0
and e.wait_type NOT IN (‘LAZYWRITER_SLEEP’, ‘SQLTRACE_BUFFER_FLUSH’
, ‘SOS_SCHEDULER_YIELD’ ,’DBMIRRORING_CMD’, ‘BROKER_TASK_STOP’
, ‘CLR_AUTO_EVENT’, ‘BROKER_RECEIVE_WAITFOR’, ‘WAITFOR’
, ‘SLEEP_TASK’, ‘REQUEST_FOR_DEADLOCK_SEARCH’, ‘XE_TIMER_EVENT’
, ‘FT_IFTS_SCHEDULER_IDLE_WAIT’, ‘BROKER_TO_FLUSH’, ‘XE_DISPATCHER_WAIT’
, ‘SQLTRACE_INCREMENTAL_FLUSH_SLEEP’)
order by (e.wait_time_ms – s.wait_time_ms) desc
–clean up table
delete from wait_stats
where snap_time < @current_snap_time
go
exec snap_waits
/* requests executing on the system
*****************************************************************/
select r.session_id
,blocking_session_id
,wait_type
,wait_time
,wait_resource
,r.status
,r.cpu_time
,r.total_elapsed_time
,r.reads
,s.reads [session reads]
,s.logical_reads [session logical reads]
,r.writes
,r.logical_reads
–,r.scheduler_id
,s.host_name
,qt.dbid
,qt.objectid
,substring(substring(qt.text,r.statement_start_offset/2+1,
(case when r.statement_end_offset = -1
then len(convert(nvarchar(max), qt.text)) * 2
else r.statement_end_offset end – r.statement_start_offset)/2)
, 1, 255) –substring
as statement_text
–,qp.query_plan
from sys.dm_exec_requests r
inner join sys.dm_exec_sessions s on (s.session_id = r.session_id)
cross apply sys.dm_exec_sql_text(sql_handle) as qt
cross apply sys.dm_exec_query_plan (plan_handle) as qp
where r.session_id > 50
–and r.session_id = 55
order by r.total_elapsed_time desc –r.status — r.scheduler_id, r.status, r.session_id
/* Top statements by total CPU time
************************************************/
SELECT TOP 25
SUBSTRING(qt.text,qs.statement_start_offset/2+1,
(case when qs.statement_end_offset = -1
then len(convert(nvarchar(max), qt.text)) * 2
else qs.statement_end_offset end -qs.statement_start_offset)/2)
as statement_text,
–substring (qt.text , 1, 512) as batch_text,
qs.total_worker_time/qs.execution_count as average_cpu_time,
qs.total_elapsed_time/qs.execution_count as average_elapsed_time,
qs.total_logical_reads/qs.execution_count as average_logical_reads,
qs.total_logical_writes/qs.execution_count as average_logical_writes,
qs.execution_count,
qs.plan_generation_num,
qs.total_worker_time,
qs.total_elapsed_time,
cast((cast(qs.total_worker_time as decimal) / cast(qs.total_elapsed_time as decimal) * 100) as int) as cpu_vs_elapsed_percentage,
qs.total_logical_reads,
qs.total_logical_writes,
db_name(qt.dbid) as [database name],
qs.plan_handle,
qt.objectid
,qp.query_plan
FROM sys.dm_exec_query_stats qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) as qt
cross apply sys.dm_exec_query_plan(qs.plan_handle) as qp
–order by qs.total_logical_reads/qs.execution_count desc
–ORDER BY total_logical_reads desc
–ORDER BY total_elapsed_time desc
ORDER BY total_worker_time desc
–115982764 (with lock)
–31250708
/* Snapshot the current spinlock stats and store so that this can be compared over a time period
Return the statistics between this point in time and the last collection point in time.*/
use tempdb
go
if exists (select * from sys.objects where name = ‘snap_spins’)
drop procedure snap_spins
go
create procedure snap_spins
as
declare @current_snap_time datetime
declare @previous_snap_time datetime
set @current_snap_time = GETDATE()
if not exists(select name from tempdb.sys.sysobjects where name like ‘spin_waits%’)
create table spin_waits
(
lock_name varchar(128)
,collisions bigint
,spins bigint
,sleep_time bigint
,backoffs bigint
,snap_time datetime
)
–capture the current stats
insert into spin_waits
(
lock_name
,collisions
,spins
,sleep_time
,backoffs
,snap_time
)
select name
,collisions
,spins
,sleep_time
,backoffs
,@current_snap_time
from sys.dm_os_spinlock_stats
select top 1 @previous_snap_time = snap_time from spin_waits
where snap_time < (select max(snap_time) from spin_waits)
order by snap_time desc
–get delta in the spin locks stats
select top 10
spins_current.lock_name
, (spins_current.spins – spins_previous.spins) as spins
,(spins_current.backoffs – spins_previous.backoffs) as backoffs
, (spins_current.collisions – spins_previous.collisions) as collisions
, (spins_current.sleep_time – spins_previous.sleep_time) as sleep_time
, spins_previous.snap_time as [start_time]
, spins_current.snap_time as [end_time]
, DATEDIFF(ss, @previous_snap_time, @current_snap_time) as [seconds_in_sample]
from spin_waits spins_current
inner join (
select * from spin_waits
where snap_time = @previous_snap_time
) spins_previous on (spins_previous.lock_name = spins_current.lock_name)
where
spins_current.snap_time = @current_snap_time
and spins_previous.snap_time = @previous_snap_time
and spins_current.spins > 0
order by (spins_current.spins – spins_previous.spins) desc
–clean up table
delete from spin_waits
where snap_time < @current_snap_time
go
exec snap_spins
/* Snapshot the current file stats and store them so that they can be compared over a time period
Return the statistics between this point in time and the last collection point in time.
This uses a persisted table in tempdb. After the needed data is captured, drop this table.
use tempdb
go
drop table _iostats_
*/
use tempdb
go
if not exists(select name from tempdb.sys.sysobjects where name like ‘_iostats_’)
create table _iostats_
(
database_id int
,file_id int
,file_guid uniqueidentifier
,num_of_bytes_read bigint
,num_of_bytes_written bigint
,num_of_reads bigint
,num_of_writes bigint
,io_stall_write_ms bigint
,io_stall_read_ms bigint
,size_on_disk_bytes bigint
,physical_name nvarchar(260)
,type_desc nvarchar(60)
,snap_time datetime
)
declare @current_snap_time datetime
declare @previous_snap_time datetime
set @current_snap_time = GETDATE()
insert into _iostats_ (
database_id
,file_id
,file_guid
,num_of_bytes_read
,num_of_bytes_written
,num_of_reads
,num_of_writes
,io_stall_write_ms
,io_stall_read_ms
,size_on_disk_bytes
,physical_name
,type_desc
,snap_time
)
select
vfs.database_id
,vfs.file_id
,mf.file_guid
,vfs.num_of_bytes_read
,vfs.num_of_bytes_written
,vfs.num_of_reads
,vfs.num_of_writes
,vfs.io_stall_write_ms
,vfs.io_stall_read_ms
,vfs.size_on_disk_bytes
,mf.physical_name
,mf.type_desc
,@current_snap_time
from sys.dm_io_virtual_file_stats(null, null) as vfs
join sys.master_files as mf on vfs.database_id = mf.database_id and vfs.file_id = mf.file_id
where vfs.database_id > 4 or vfs.database_id = 2
order by vfs.database_id, vfs.file_id
select top 1 @previous_snap_time = snap_time from _iostats_
where snap_time < (
select max(snap_time) from _iostats_) order by snap_time desc
print ‘Current snap time: ‘ + convert(varchar(32), @current_snap_time)
print ‘Previous snap time: ‘ + convert(varchar(32), @previous_snap_time)
declare @tick_count_between_snaps bigint
set @tick_count_between_snaps = DATEDIFF(ms, @previous_snap_time, @current_snap_time)
print ‘@tick_count_between_snaps: ‘ + convert(varchar(32), @tick_count_between_snaps)
select
time_in_sample_secs = DATEDIFF(s, @previous_snap_time, @current_snap_time)
,db = db_name(iostats_now.database_id)
,iostats_now.physical_name
,iostats_now.type_desc
,iostats_now.file_id
,(iostats_now.num_of_bytes_read – iostats_lastsnap.num_of_bytes_read) num_of_bytes_read
,(iostats_now.num_of_bytes_written – iostats_lastsnap.num_of_bytes_written) num_of_bytes_written
,(iostats_now.num_of_reads – iostats_lastsnap.num_of_reads) num_of_reads
,(iostats_now.num_of_writes – iostats_lastsnap.num_of_writes) num_of_writes
,avg_read_IOPs = case when (iostats_now.num_of_reads – iostats_lastsnap.num_of_reads) = 0 then 0 else ((iostats_now.num_of_reads – iostats_lastsnap.num_of_reads) /(@tick_count_between_snaps/1000)) end
,avg_read_bytes_sec = case when (iostats_now.num_of_bytes_read – iostats_lastsnap.num_of_bytes_read) = 0 then 0 else ((iostats_now.num_of_bytes_read – iostats_lastsnap.num_of_bytes_read)/(@tick_count_between_snaps/1000)) end
,avg_read_stall_ms = case when (iostats_now.num_of_reads – iostats_lastsnap.num_of_reads) = 0 then 0 else ((iostats_now.io_stall_read_ms – iostats_lastsnap.io_stall_read_ms) /(iostats_now.num_of_reads – iostats_lastsnap.num_of_reads)) end
,avg_read_size = case when (iostats_now.num_of_reads – iostats_lastsnap.num_of_reads) = 0 then 0 else ((iostats_now.num_of_bytes_read – iostats_lastsnap.num_of_bytes_read)/(iostats_now.num_of_reads – iostats_lastsnap.num_of_reads)) end
,avg_write_IOPs = case when (iostats_now.num_of_writes – iostats_lastsnap.num_of_writes) = 0 then 0 else ((iostats_now.num_of_writes – iostats_lastsnap.num_of_writes) /(@tick_count_between_snaps/1000)) end
,avg_write_bytes_sec = case when (iostats_now.num_of_bytes_written – iostats_lastsnap.num_of_bytes_written) = 0 then 0 else ((iostats_now.num_of_bytes_written – iostats_lastsnap.num_of_bytes_written)/(@tick_count_between_snaps/1000)) end
,avg_write_stall_ms = case when (iostats_now.num_of_writes – iostats_lastsnap.num_of_writes) = 0 then 0 else ((iostats_now.io_stall_write_ms – iostats_lastsnap.io_stall_write_ms) /(iostats_now.num_of_writes – iostats_lastsnap.num_of_writes)) end
,avg_write_size = case when (iostats_now.num_of_writes – iostats_lastsnap.num_of_writes) = 0 then 0 else ((iostats_now.num_of_bytes_written – iostats_lastsnap.num_of_bytes_written)/(iostats_now.num_of_writes – iostats_lastsnap.num_of_writes)) end
,iostats_now.size_on_disk_bytes
,filegrowth = iostats_now.size_on_disk_bytes – iostats_lastsnap.size_on_disk_bytes
,iostats_now.file_guid
from _iostats_ as iostats_now
inner join
(select * from _iostats_
where snap_time = @previous_snap_time)
iostats_lastsnap on ( –iostats_lastsnap.file_guid = iostats_now.file_guid
iostats_lastsnap.file_id = iostats_now.file_id AND iostats_lastsnap.database_id = iostats_now.database_id
)
where (iostats_now.database_id > 4 or iostats_now.database_id = 2)
and iostats_now.snap_time = @current_snap_time
and iostats_lastsnap.snap_time = @previous_snap_time
order by iostats_now.database_id asc, iostats_now.file_id asc
–clean up
delete from _iostats_
where snap_time = @previous_snap_time
–drop table _iostats_
In addition to the scripts provided earlier, you should capture the following performance monitor counters over the same time period.
\LogicalDisk(*)\Avg. Disk Bytes/Read
\LogicalDisk(*)\Avg. Disk Bytes/Write
\LogicalDisk(*)\Avg. Disk Queue Length
\LogicalDisk(*)\Avg. Disk sec/Read
\LogicalDisk(*)\Avg. Disk sec/Write
\LogicalDisk(*)\Current Disk Queue Length
\LogicalDisk(*)\Disk Read Bytes/sec
\LogicalDisk(*)\Disk Reads/sec
\LogicalDisk(*)\Disk Write Bytes/sec
\LogicalDisk(*)\Disk Writes/sec
\Memory\Available MBytes
\Memory\Free System Page Table Entries
\Memory\Pages/sec
\Network Interface(*)\*
\Process(*)\*
\Processor(*)\% Privileged Time
\Processor(*)\% Processor Time
\SQLServer:Availability Replica(*)\*
\SQLServer:Access Methods\*
\SQLServer:Buffer Manager\*
\SQLServer:Buffer Node\*
\SQLServer:Databases(*)\*
\SQLServer:Database Replica(*)\*
\SQLServer:General Statistics\*
\SQLServer:Latches\*
\SQLServer:Locks(_Total)\Average Wait Time (ms)
\SQLServer:Locks(_Total)\Lock Requests/sec
\SQLServer:Memory Manager\*
\SQLServer:Plan Cache\*
\SQLServer:Wait Statistics\*
\SQLServer:SQL Statistics\*
\System\Context Switches/sec
\System\Processor Queue Length
Note that we also recommend that you capture all replication counters when using replication.
The logman utility can be used to create the Performance Monitor counter set. Simply copy the above counter definition listed here into a file, such as counters.txt, and then run the following at the command prompt:
logman create counter <counterfilename> –cf counters.txt
For a deeper understanding of Azure Virtual Machines and their capabilities, refer to the Microsoft Azure Virtual Machines Documentation.
To explore more about SQL Server performance tuning specifically in Azure environments, visit SQL Server Performance Tuning in Azure.