Jekyll2023-12-25T13:36:01+00:00https://www.rupeshtiwari.com/feed.xmlRupesh Tiwari - Founder of Fullstack MasterRupesh is a world renowned Senior Software Architect & Trainer. Expert on Service-Oriented Architecture and Domain-Driven Design & Web Development. Make your next project a success or save a late project.Rupesh TiwariMandatory Algorithms for Coding Interview2023-01-14T00:00:00+00:002023-01-14T00:00:00+00:00https://www.rupeshtiwari.com/mandatory-algorithms-for-coding-interview<blockquote>
<p>Basic algorithms that you must know if you want to crack your coding interview. In case you are starting to prepare for coding practice then make sure you learn all of these important algorightms. Get the <a href="https://codepen.io/collection/BNaddx">source code for basic algorithms in JavaScript here</a></p>
</blockquote>
<h2 id="binary-search-in-sorted-array">Binary Search in Sorted Array</h2>
<table>
<tbody>
<tr>
<td>**O(log(n) time</td>
<td>O(1) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://imgur.com/vGpwSKt.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="rNmBqWQ" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/rNmBqWQ">
Binary Search Tree Implementation</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="kadanes-algorithm">Kadane’s Algorithm</h2>
<p><img src="https://imgur.com/i1sGon1.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="bGrBbZr" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/bGrBbZr">
Kadane's Algorithm</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="moores-voting-algorithm">Moore’s Voting Algorithm</h2>
<p><img src="https://i.imgur.com/JT3TvCV.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="YzVLbjK" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/YzVLbjK">
Moore's Voting Algorithm | Majority Sum | Array</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="reverse-linked-list">Reverse Linked List</h2>
<table>
<tbody>
<tr>
<td>**O(n) time</td>
<td>O(1) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/MbFK2KZ.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/sTbk2kj.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="jOmNvRw" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/jOmNvRw">
Reverse Linked List Algorithm</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="flyods-cycle-detection-algorithm">Flyod’s Cycle Detection Algorithm</h2>
<p><img src="https://i.imgur.com/SzgerO1.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/fMuMGAp.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="xxLRxzV" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/xxLRxzV">
Untitled</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="sorting-algorithms">Sorting Algorithms</h2>
<h3 id="merge-sort">Merge sort</h3>
<blockquote>
<p>In Merge sort since I need 2 Aux arrays. And it requires more space to solve merge sorting problem. Therefore, I will prefer <strong>Linked List</strong> datastructure. Because, in linkedlist to store values, I do not need contigous memory slots. Hence optimizing memory usage.</p>
</blockquote>
<table>
<tbody>
<tr>
<td>**O(n log (n)) time</td>
<td>O(1) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/wozvfAe.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js" data-slug-hash="WNjegBp" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/WNjegBp">
Merge Sort Algorithm</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h3 id="quick-sort">Quick Sort</h3>
<table>
<tbody>
<tr>
<td>**O(n log (n)) time</td>
<td>O( log (n) ) and wrost case O(n) space**</td>
</tr>
</tbody>
</table>
<blockquote>
<p>Quicksort is best fit for sorting <strong>Arrays</strong> since it is completely in-place sorting. No auxilary arrays are requried. So no need to store contigous memory slots on computer.</p>
</blockquote>
<p><img src="https://i.imgur.com/f3zTZNq.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="OJpBYKz" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/OJpBYKz">
Quick Sort In-place Implementation: Answer</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h3 id="partition-logic">Partition Logic</h3>
<ul>
<li>All the element lesser than pivot are pushed to the left of the partition index (use swap)</li>
</ul>
<p><img src="https://i.imgur.com/XrO2FKz.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/iI22dTH.png" alt="" class="full" /></p>
<h2 id="tree-traversal">Tree Traversal</h2>
<h3 id="breadth-first-traversalbft">Breadth-First Traversal(BFT)</h3>
<p><strong>Whenever tree is Sparse use BFT</strong></p>
<table>
<tbody>
<tr>
<td>**O(n) time</td>
<td>O(n) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/yW3E6Ph.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="XWRroaz" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/XWRroaz">
Breadth First Traversal | Binary Tree</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h3 id="depth-first-traversal-dft">Depth-First Traversal (DFT)</h3>
<table>
<tbody>
<tr>
<td>**O(n) time</td>
<td>O(n) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/mbFltXf.png" alt="" class="full" /></p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="ExmYGze" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/ExmYGze">
Depth-First Traversal | Binary Tree</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h2 id="binary-search-tree-bst">Binary Search Tree (BST)</h2>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="rNmBqWQ" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/rNmBqWQ">
Binary Search Tree Implementation</a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h3 id="insert-in-binary-search-tree-bst">Insert in Binary Search Tree (BST)</h3>
<table>
<tbody>
<tr>
<td>**O(Log(n)) time</td>
<td>Space O(1)**</td>
</tr>
</tbody>
</table>
<p>For 1 insert operation, avg case is O(lgn) and worst case is O(n)
For n insert operations, avg case is O(nlgn) and worst case is O(n^2)</p>
<p><img src="https://i.imgur.com/bcBZn35.png" alt="" class="full" /></p>
<h3 id="remove-node-from-bst">Remove Node from BST</h3>
<table>
<tbody>
<tr>
<td>**O(log n) time</td>
<td>space O(1)**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/KCKZohE.png" alt="" class="full" /></p>
<h3 id="search-in-bst">Search in BST</h3>
<p><strong>O(log(n)) time | O(1)</strong>
<img src="https://i.imgur.com/rb6qlIB.png" alt="" class="full" /></p>
<h3 id="min-max-and-height-in-bst">Min Max and Height in BST</h3>
<p><img src="https://i.imgur.com/3VEHOE1.png" alt="" class="full" /></p>
<script async="" src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>
<h2 id="heap-implementation">Heap implementation</h2>
<p>In the interview no one will ask you to implement heap. This excercise is for your coding skills imporvement. In the interview, you can use array instead of heap and explain the concept.</p>
<p class="codepen" data-height="300" data-default-tab="js,result" data-slug-hash="OJmVjgB" data-user="rupeshtiwari" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;">
<span>See the Pen <a href="https://codepen.io/rupeshtiwari/pen/OJmVjgB">
Heaps: Implement Heap </a> by Rupesh Tiwari (<a href="https://codepen.io/rupeshtiwari">@rupeshtiwari</a>)
on <a href="https://codepen.io">CodePen</a>.</span>
</p>
<h3 id="insert-node-in-the-heap">Insert node in the Heap</h3>
<table>
<tbody>
<tr>
<td>**O(log(n) time</td>
<td>O(1) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/H1Hm4Pe.png" alt="" class="full" /></p>
<h3 id="revove-root-node-from-the-heap">Revove root node from the Heap</h3>
<table>
<tbody>
<tr>
<td>**O(log(n) time</td>
<td>O(1) space**</td>
</tr>
</tbody>
</table>
<p><img src="https://i.imgur.com/5gmtiBx.png" alt="" class="full" /></p>
<h2 id="reference">Reference</h2>
<ul>
<li><a href="https://www.youtube.com/user/mycodeschool">My Code School Youtube channel</a></li>
<li><a href="https://codepen.io/collection/BNaddx">Source code of mandatory algorithms in JavaScript</a></li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariBasic algorithms that you must know if you want to crack your coding interview. In case you are starting to prepare for coding practice then make sure you learn all of these important algorightms. Get the source code for basic algorithms in JavaScript hereApplication and Service Principal Objects in Azure2023-01-07T00:00:00+00:002023-01-07T00:00:00+00:00https://www.rupeshtiwari.com/application-and-service-principal-objects-in-azure<p>Application and Service Principal Objects in Azure</p>
<hr />
<blockquote>
<p>In azure cloud in order to protect your resources like web applications you must register them in Azure Active Directory. After registering an application you get service principal. All of your users registered in Azure AD have user principal as security key. .</p>
</blockquote>
<h2 id="application-registration-in-azure-active-directory">Application Registration in Azure Active Directory</h2>
<p>You register your application in Azure AD tenant to manage identity & access by creating identity configuration of your application. While registration you must specify:</p>
<ol>
<li>URL of your application</li>
<li>URL to redirect after authentication</li>
<li>URI to identify your application</li>
</ol>
<p>The Azure AD tenant where you register your application is called HOME TENANT. You can choose to give access to either single or multiple tenants for your application. You can register multiple apps with the same name however they will must have unique application id.
<img src="https://i.imgur.com/zPk3Sth.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/gfqUETE.png" alt="" class="full" /></p>
<h2 id="what-is-an-application-object">What is an Application Object?</h2>
<p>After you register an application in Azure AD Home Tenant. It creates a globally unique instance of the application known as Application object. You get globally unique Id (application or client Id ) for your registered application. After registering application you also get service principal auto created (object Id)</p>
<p><img src="https://i.imgur.com/YtniiGk.png" alt="" class="full" /></p>
<p>You can add secrets or certificates & scope to your application. You can use an application object to create multiple service principal objects for your application. Service Principal objects are the same as application instances.</p>
<p><img src="https://i.imgur.com/ZeblZYd.png" alt="" class="full" /></p>
<p>If you use your application in multiple tenants then a new service principal is created in that tenant. However application objects remain single in its home tenant as global identity.
<img src="https://i.imgur.com/qMvFl9P.png" alt="" class="full" /></p>
<p>There are some static properties in application objects that is always common to all application instances or service principals.</p>
<p>Application object describes 3 different aspects of an application as following:</p>
<ol>
<li>How the service can issue tokens in order to access application</li>
<li>Resources that application might access</li>
<li>The actions that the application can take</li>
</ol>
<p>Application Entity defines the schema for an application object’s properties.</p>
<h2 id="what-is-a-service-principal-object">What is a Service Principal Object?</h2>
<p>To access resources that are secured by an Azure AD tenant, the entity that requires access must be represented by a security principal. This requirement is true for both users (user principal) and applications (service principal). The security principal defines the access policy and permissions for the user/application in the Azure AD tenant. This enables core features such as authentication of the user/application during sign-in, and authorization during resource access.</p>
<p>There are three types of service principal: application, managed identity, and legacy.</p>
<p>The first type of service principal is the local representation, or application instance, of a global application object in a single tenant or directory. A service principal is created in each tenant where the application is used and references the globally unique app object. The service principal object defines what the app can actually do in the specific tenant, who can access the app, and what resources the app can access.</p>
<p>When an application is given permission to access resources in a tenant (upon registration or <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/developer-glossary#consent">consent</a>), a service principal object is created. When using the portal, a service principal is created automatically when you register an application.</p>
<p>The second type of service principal is used to represent a <a href="https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview">managed identity</a>. Managed identities eliminate the need for developers to manage credentials. Managed identities provide an identity for applications to use when connecting to resources that support Azure AD authentication. When a managed identity is enabled, a service principal representing that managed identity is created in your tenant. Service principals representing managed identities can be granted access and permissions, but cannot be updated or modified directly.</p>
<p>The third type of service principal represents a legacy app (an app created before app registrations were introduced or created through legacy experiences). A legacy service principal can have credentials, service principal names, reply URLs, and other properties which are editable by an authorized user, but does not have an associated app registration. The service principal can only be used in the tenant where it was created.</p>
<p>The Enterprise applications blade in the portal is used to list and manage the service principals in a tenant.</p>
<h2 id="relationship-between-application-objects-and-service-principals">Relationship between application objects and service principals</h2>
<p>The application object is the global representation of your application for use across all tenants, and the service principal is the local representation for use in a specific tenant.</p>
<p>The application object serves as the template from which common and default properties are derived for use in creating corresponding service principal objects. An application object therefore has a 1:1 relationship with the software application, and a 1:many relationship with its corresponding service principal object(s).</p>
<p>A service principal must be created in each tenant where the application is used, enabling it to establish an identity for sign-in and/or access to resources being secured by the tenant. A single-tenant application has only one service principal (in its home tenant), created and consented for use during application registration. A multi-tenant application also has a service principal created in each tenant where a user from that tenant has consented to its use.</p>
<h2 id="consequences-of-modifying-and-deleting-applications">Consequences of modifying and deleting applications</h2>
<p>Any changes that you make to your application object are also reflected in its service principal object in the application’s home tenant only (the tenant where it was registered). This means that deleting an application object will also delete its home tenant service principal object. However, restoring that application object will not restore its corresponding service principal. For multi-tenant applications, changes to the application object are not reflected in any consumer tenants’ service principal objects, until the access is removed through the <a href="https://myapps.microsoft.com/">Application Access Panel</a> and granted again.</p>
<h2 id="example-of-service-principal-and-application-object">Example of Service principal and Application object</h2>
<p>The following diagram illustrates the relationship between an application’s application object and corresponding service principal objects, in the context of a sample multi-tenant application called HR app. There are three Azure AD tenants in this example scenario:</p>
<ul>
<li>Adatum - The tenant used by the company that developed the HR app</li>
<li>Contoso - The tenant used by the Contoso organization, which is a consumer of the HR app</li>
<li>Fabrikam - The tenant used by the Fabrikam organization, which also consumes the HR app</li>
</ul>
<p><img src="https://i.imgur.com/zpgHTR8.png" alt="" class="full" /></p>
<h2 id="how-to-create-a-service-principal">How to create a Service Principal?</h2>
<p>You can not create service principal from Azure portal as of November, 2021. Check Microsoft document for latest update.</p>
<p>Service principal is an identity created for use in application, hosted service and automated tools to access Azure resources. You should always use service principal for automated tools rather than login with user identity.</p>
<p><img src="https://i.imgur.com/7uSpO0r.png" alt="" class="full" /></p>
<p>When you create service principal it gives you credentials that you need while in use of service principal. If you don’t want to manage credentials try creating managed identity. If you don’t have a password for the service principal then you can use the reset service principal credential option.</p>
<p>By default when we create a service principal it assigns a contributor role on subscription scope. You can use `New-AzADServicePrincipal` powershell cmdlet to create a new service principal.</p>
<h2 id="service-principal-authentication-types">Service Principal Authentication Types</h2>
<p>There are 2 types of authentications for service principal:</p>
<ol>
<li>Password-based (application secret)</li>
<li>Certificate-based (Recommended)</li>
</ol>
<h3 id="password-based-authentication">Password-Based Authentication</h3>
<p>When you create a service principal for password-based. It will return an auto generated password. You need that password to authenticate. By default service principal gets read and write access at subscription scope. For sign In using service principal you need tenant Id. Use <code class="language-plaintext highlighter-rouge">New-AZADServicePrincipal -DisplayName Web</code> to create a new one.</p>
<p><img src="https://i.imgur.com/iJ8nGgm.png" alt="" class="full" /></p>
<h3 id="certificate-based-authentication">Certificate-based Authentication</h3>
<p>In order to create a certificate-based service principal you need to pass a base-64 encoded ASCII string of public certificates. Once it is created it will return Id and displayname.</p>
<h2 id="references">References</h2>
<ol>
<li><a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals#application-objec">https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals#application-object</a></li>
<li><a href="https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-5.9.0">https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-5.9.0</a></li>
<li><a href="https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-5.9.0#manage-service-principal-roles">https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-5.9.0#manage-service-principal-roles</a></li>
</ol>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariApplication and Service Principal Objects in AzureHow to improve Reliability in the cloud?2021-12-12T00:00:00+00:002021-12-12T00:00:00+00:00https://www.rupeshtiwari.com/how-to-improve-reliability-in-the-cloud<blockquote>
<p>Did you get a phone call from your customer saying they want to improve their application reliability while you move their workload to the cloud? If yes then continue reading this article. I will use Microsoft Azure to explain the Reliability concept and its improvement steps. You can certainly do the same in Google Cloud or Amazon AWS.</p>
</blockquote>
<h2 id="what-is-reliability">What is Reliability?</h2>
<p>Well reliability is a derived concept. Basically you want your application to be available for your customer. Also you should ask what latency is acceptable for your application. If your application is slower or normal, your application users should not keep retrying to finish any transaction. How should your application fidelity be while your application is slow and not rendering properly. So in order to define reliability of your application you have to answer how much your application should be Available and what is your Latency that you want for your application.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Reliability = Availability + Latency
</code></pre></div></div>
<p>So basically reliability is something your Business has to define. You must talk to your customer and understand what kind of business they are running, what problem they want to solve, who are the target users for the application.</p>
<h2 id="how-to-increase-reliability">How to increase Reliability?</h2>
<p>Reliability of your workload is a shared responsibility between you and your cloud provider. Platform Reliability such as datacenter, network and other hardware infrastructure is your cloud provider responsibility. However, application reliability is customer responsibility.</p>
<p>In order to increase application reliability you must do below:</p>
<ol>
<li>Choosing and Configuring correct cloud building blocks.</li>
<li>Your application must be cloud aware. Like your code should have Retry logic to make sure that application is handling transient failures which is common in the cloud.</li>
</ol>
<h2 id="right-building-blocks-to-increase-reliability">Right Building Blocks to increase Reliability</h2>
<p>Make sure you choose the right infrastructure building blocks in the cloud to protect their reliability impacts. You have to isolate the concerns and identify them like below:</p>
<ul>
<li>Isolated VM failure such as OS, disk HDD(hard disk drive) issue.</li>
<li>Hardware failure such as server rack issues.</li>
<li>Entire datacenter failure such as power/network issues.</li>
<li>Entire Region failure such as natural disasters.</li>
</ul>
<p>In order to protect VM disk and OS issues you need to use Premium Storage from Azure Cloud. To protect from hardware failure you must create a VM inside the Availability sets. To protect from entire data center failure you must consider deploying your workload in multiple Availability Zones. Also to protect from natural disasters make sure you deploy your workload in another region as well. So in summary you have to choose correct building blocks as per your requirement to improve reliability.</p>
<p><img src="https://lh6.googleusercontent.com/NmAPF2RRL9cWnGNk2trA4CDiJYP1T94JKw6eCDdIaiMh6cK-cDmxK5L-kFXm6arB43AGSYdXRrJ9NknkOoNxp6xavK_8DQohHvWjJDwkYWthLHm59aQYzUcNkjcjUSh7sw1cvqQy" alt="" /></p>
<h2 id="how-to-improve-reliability-of-an-existing-workload">How to Improve Reliability of an Existing Workload?</h2>
<p>Start backup for your virtual machine, databases that you can enable for existing resources in Azure cloud. However, if your VMs are not in the Availability set then you can not put them in the availability set for existing resources. You have to do workaround and re-build some of them and put them in Availability sets again.</p>
<p>In order to improve reliability of existing workload you must follow below steps on each use case:</p>
<ol>
<li>Find Key components for the critical use case (which business is more concern)</li>
<li>Failure Mode Analysis for the critical use case</li>
<li>Calculate Composite SLA for the critical use case</li>
<li>Improve Composite SLA to meet business requirement</li>
</ol>
<h2 id="failure-mode-analysis-technique-for-reliability">Failure Mode Analysis Technique for Reliability </h2>
<p>First you should talk to customers to find out which business capability they want to be up for what time. Example the purchase application written as asp.net needs to be up for 99.99 and 4 min per month down time. If business defines this then next you must learn their existing workload design consideration to achieve 99.99 SLA.</p>
<p>Designing for failure is the philosophy to take while designing applications in the cloud. Exploring what happens if something fails? Take the architecture, look at the portions of the application piece by piece and apply failure mode analysis. Break this down into your solution as a series of key processes. Like in your application, the customer goes and searches for an item to buy.</p>
<h2 id="step-1-find-the-key-components-of-the-business-use-case">Step 1: Find the key components of the business use case</h2>
<p>So now let’s search what component has to be working to make search possible?</p>
<p>Fo below architecture example in order to make sure Search is working below components must be up and running</p>
<ul>
<li>AKS with .net application</li>
<li>Products service</li>
<li>Cosmos DB</li>
</ul>
<p><img src="https://lh4.googleusercontent.com/f4_vi4hkaP9Mm_CegNb3EIdZJi4sdcP6M5V4N4UN3hQx9xv6d2EzAgXgfGsl_Pt2fp2XjeWY8a691sTLYNE46L-XBLkWtaMp8euwTTKE3d-WfPKCQvTTDn8VYk1vFvsZORVVDv4j" alt="" /></p>
<h2 id="step-2-failure-mode-analysis">Step 2: Failure Mode Analysis</h2>
<p>Try to write down each component failure mode and their effect, impact and chances and work with your customer to learn more. This is a collaborative effort. Solutions Architect and Customer together must do this to come up with correct decisions. For search use cases you want to check what happens if AKS is unavailable, Website is slow, CosmosDB unavailable for read and update use case.</p>
<ul>
<li><strong>AKS</strong> is unavailable so we have to change the SLA to 99.95% on the API server of Kubernetes which is a controller plane. This is the SKU version we have to upgrade. In the worker plane we have a VM whose SLA is just like regular. So businesses have to think of their own requirements to opt in if they want to upgrade the SLA or kubernetes.</li>
<li><strong>Web site</strong> is down, because of fluctuation of load due to holiday session. Probability is unknown. So you must use a monitoring tool setup alert to learn more about this kind of problem. LOAD OVER TIME can be only learnt by monitoring and it’s trends.</li>
<li><strong>Cosmos DB</strong> can not read or update. Cosmos DB has 99.99% SLA for a single region. Cosmos DB can not be updated so customers can not see the latest catalog, it is also covered by 99.99% SLA out of the box by Azure cloud.</li>
</ul>
<p><img src="https://lh5.googleusercontent.com/iu8LobbMlbCm015XHh-IK0LDNnCgnpcy4fci8U2yyfL0CqeXKbllaoobPBFa3aUnRny_HuJevOw2g3W-XlZyuKOn7l4j2ogKzdhFrWhEvco1cz9ALewb8MHlbKStECxoTR3XOLKh" alt="" /></p>
<h2 id="step-3-calculate-composite-sla">Step 3: Calculate Composite SLA</h2>
<p>AKS SLA is 99.95% and Cosmos DB SLA is 99.99%</p>
<p>So overall composite SLA of AKS and Cosmos DB is 99.99 x 99.95 = <strong>99.94%</strong></p>
<p><img src="https://lh4.googleusercontent.com/GCIAG58M95wm2POxLK8YbpqaPsmb_V-znzAPJ7-hcE_00BWNV3l4WTNCv7IjuHLiERLMkxy0-8nfVEDsdAVf776lQj3I15NdqVRF1k5XiXdBjhu2KCMfO-XVmqoyi7p07swRy3Rq" alt="" /></p>
<h2 id="step-4-how-to-increase-composite-sla">Step 4: How to increase Composite SLA</h2>
<p>To increase the composed availability add redundancy to the solution. Let’s make 2 AKS, if one breaks the other will work. You can put them in multiple Azure Region East/South USA. They have to be identical. Deployment of the cluster and the containers would have to be deployed in an automated way to keep them identical. IAC (Infrastructure as Code) should be in a repository and used to deploy your resources. Since AKS is in multiple regions you need DB in both regions. Cosmos DB has a multimaster database you can use in 2 regions so any change you do in one region is synchronized in both regions out of the box. You have to use Microsoft <a href="https://docs.microsoft.com/en-us/azure/frontdoor/front-door-overview">Front Door</a> to route between multiple regions.</p>
<ul>
<li>AKS (Azure Kubernetes Service) with redundant will give you SLA 99.99% per region with multiple region SLA is 99.999975%</li>
<li>Front Door SLA is 99.99%</li>
<li>Cosmos DB with multiple writable region SLA 99.999%</li>
</ul>
<p><strong>Composite SLA</strong> = 0.99999975 x 0.9999 x 0.99999 = <strong>99.99%</strong></p>
<p><img src="https://lh3.googleusercontent.com/FfWkyWrjFJtX4F-lFVnBJWvy-8VmvLHrKxGjbibArAZ6VmeaZQsvTysSbFuG6B4TR8hSsL8AkGLzvo7vhk6gHPlnkQ-ajJH7gSlYKBlvVkjCjSps-HPT62EDogXQrqSyW-kWkSTJ" alt="" /></p>
<h2 id="how-to-know-if-something-is-wrong-in-your-web-app">How to know if something is wrong in your web app?</h2>
<p>Create alert rules specify conditions and for actions you can alert humans with sms, or create action groups and execute some azure functions.</p>
<p><img src="https://lh3.googleusercontent.com/_D_EY_Jkd2qcVom9MCo4xRthm8E0rTF2iOW9ib_kLSMr3s1f1mWaPaFZ66AhVNEObFywS1qApOPG6PtaDwaks3ZJsU-5NJ33jT0j0OJFvF4ucF9cze8Gul6AfTyiigzA0d3g25BA" alt="" /></p>
<h2 id="enable-vm-replication-for-improved-reliability">Enable VM Replication for improved Reliability</h2>
<p>In addition to backup lots of customers are focused on disaster recovery and making sure the workloads are redundant and not just backed up. Meaning that if we had a planned or unplanned type of outage. Planned means upgrades to different applications, or we are testing our disaster recovery or may be unplanned like some natural disaster or things of nature. So introduce resilience in the form of being able to access backups that are in another region and to make those come online as quickly as possible to make sure that we reduce downtime to the extent possible.</p>
<h2 id="upgrade-vm-to-premium-disks-to-increase-reliability">Upgrade VM to Premium Disks to increase Reliability</h2>
<p>For production workloads, especially Big Data, SQL and other transactional DB upgrades to premium disk instead of standar disk offering. Premium disks also offer bursting capabilities.</p>
<p>Burst means performance on heavy load.</p>
<ul>
<li>On-demand bursting: you pay whenever you have burst. When you have an inconsistent workload.</li>
<li>Credit-based bursting: Harvest or credit yourself with additional bursting capability when you are operating below the maximum thresholds that we have. In this case, you’re able to burn those credits by bursting when you need it without any other cost implications. So you accrue your bursting credits when you are not using them and use it when you need.</li>
</ul>
<h2 id="upgrade-vm-to-managed-disks-to-increase-reliability">Upgrade VM to Managed Disks to increase Reliability</h2>
<p>Un-Managed Disks where you have to set the size, throughput limit, and add more resources to scale are called unmanaged disks.</p>
<p>However in Managed Disk you get below processes automated for you by your cloud provider:</p>
<ul>
<li>Set the size, throughput limit, add more resource to scale handled by azure</li>
<li>Abstract the entire management layer of storage</li>
<li>Compatible with availability set and availability zones, you get 99.99% availability for production workloads.</li>
<li>Resiliency : 3 replicas across multiple hardware devices</li>
<li>Granular role based access control</li>
</ul>
<h2 id="how-do-i-know-what-i-dont-know">How do I know what I don’t Know?</h2>
<p>You have to have a long term learning process. Just wait for another disaster, unexpected event and learn. The other option is to simulate/force failures and see what happens. Say stop AKS and check if your redundancy is working. Take a more experimental model, introduce real world scenarios and see how our workload works. Game day, setup scenario, one team generating un-predictable load by un-usual way, another team observing, are we seeing alerts are coming, are we seeing redundant machine status.</p>
<h2 id="summary">Summary</h2>
<p>In order to get Reliability increased for your customers workload you must speak with them and understand their critical use case which they care most about. Follow the steps that I suggested in this article to come up with the desired reliability state defined by business by doing failure mode analysis in depth one by one for each important use case. You may have to teach your customer about premium disks for VM and their SLA improvement because of upgrading to premium disk. What is managed and unmanaged disks after migrating to managed disk resilience can be increased since it automatically replicates the disk into 3 multiple hardware devices. Consider replicating virtual machines to protect them from regional outages. Setup Azure Virtual Machine backup for improved reliability and to protect from human error or DB corruptions. Also make sure you set up alerts for your applications to send you email or create zendesk tickets whenever it is down. I hope now you have a good understanding about reliability improvement.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariDid you get a phone call from your customer saying they want to improve their application reliability while you move their workload to the cloud? If yes then continue reading this article. I will use Microsoft Azure to explain the Reliability concept and its improvement steps. You can certainly do the same in Google Cloud or Amazon AWS.Cloud Security Defense In-Depth Azure Approach2021-12-11T00:00:00+00:002021-12-11T00:00:00+00:00https://www.rupeshtiwari.com/cloud-security-defense-in-depth-azure-approach<blockquote>
<p>In a traditional data center you create one perimeter, secure them by installing firewall, WAF, SIEMS etc. and have 100% confidence that the data center is secure. However, when you migrate to cloud then your VPC or Virtual Network is not 100% secure you have to not only secure your perimeter however, inside of perimeter you have to take care of security. Since it’s a public cloud you have to be more cautious and make sure whatever workload or source code you are moving to is cloud aware. There are various best practices at each level you have to take care of in order to secure your cloud.</p>
</blockquote>
<h2 id="what-is-defense-in-depth">What is Defense in-depth? </h2>
<p>There is a concept of Defense in-depth which makes sure that at each level there is security in-place which makes your cloud data center almost 100% secure. However, it needs continuous awareness, assessments and audits. In this article let’s explore how Azure helps us to secure data centers in the cloud.</p>
<p>Defense in-depth design of azure services and capabilities to help you secure, manage and monitor your cloud data, infrastructure, compute Azure offers unified security management and advanced threat protection for your cloud, on-premise data centers or both.</p>
<p><img src="https://lh5.googleusercontent.com/vnX_FyeIYoghC9nnzuhYRmZlfep0AaHXTY0VfyH8mr2zy-JD4PSrFGyqqOSfRUGBn7WciGid8T3nGQm4ZPMdM8Qv5XYkviRwTUJ1vKUIL971w5AyxavBM14DoCiWYTBLrIhG54C1" alt="" /></p>
<p>When you want to secure your cloud you should think of securing from the physical layer till your data layer.</p>
<p>Defense In-depth security layers are as follows:</p>
<ul>
<li>Physical Security ( within your data-center )</li>
<li>Policies and Access / Identity and Access Management</li>
<li>Perimeter</li>
<li>Networking</li>
<li>Virtual Machines/Compute</li>
<li>Applications</li>
<li>Data</li>
</ul>
<h2 id="microsoft-azure-security-center">Microsoft Azure Security Center</h2>
<p>In Microsoft Azure all resources and services are designed to keep Defense in-depth in mind.</p>
<p>Azure Advisor in Azure helps us to maintain our workload to follow well-architected-framework guidelines by showing overall score. Similarly <a href="https://docs.microsoft.com/en-us/azure/security-center/defender-for-cloud-introduction">Azure Security Center</a> helps us to do defense-in-depth and it shows the all score of your cloud security health.</p>
<p><img src="https://lh5.googleusercontent.com/ruC9kWhf2WcpDIFBNy6ZpQ5KSDtnxxQbb7y4J0TYTo96YnACCHif8JFRdTReNj9ecwd-JNSmsYCKOZoBtaOa3KObRKrG6ADUVcZMEPYIWXuDXc3DgBywQtTnlSCwCEWL-plV2aNj" alt="" /></p>
<p>Security center has a lot of practical recommendations and quick fixes to keep our workload protected.</p>
<p>Azure Security center provides unified visibility, control and adaptive threat protection to the resources. So if you add any new resource they will be automatically under threat protection and detection policies. So they are secured from all network attacks.</p>
<p><img src="https://lh6.googleusercontent.com/eBAkNh4BM0mJ1f9vV2xSVDYjGY-vbxlEu3XOfTcSS6uijG4hpiTc4dwQ6UVndv6G_Tp_SmhwtzVg70LcsM-5JHki3kQGlgRUYLNrIuLjZ1C_OyswNjzrOOxtxFRA5Jhwx85pk7zR" alt="" /></p>
<p>Additionally Azure Security Center has <a href="https://docs.microsoft.com/en-us/azure/security/fundamentals/threat-detection">Intelligent Threat Detection response</a>.</p>
<p><img src="https://lh5.googleusercontent.com/RwFiOhDn3NJnVHZ1HKqeMX_k14lo5wfBzT0zqwx7vhDFGB0b3XkIrVbniuGC3K2qN0qJ6nLLz_m_gH5rdu7mKTiGqeJhOAvcmD_ze4CGShDp1uU4u8u4NAXWQqCEJB-VziQ_Dusg" alt="" /></p>
<p>Let’s understand how we would set up defense in-depth in the cloud starting from physical layer security up to data layer security.</p>
<h2 id="physical-security">Physical Security </h2>
<p>For physical security of cloud Infrastructure cloud providers are responsible. Therefore as a customer we should not be worried about securing physical data centers, racks, wire etc. Azure data center security is at high top priority. Cloud providers like Azure or AWS or Google Cloud are responsible for “Security of the Cloud” - Azure is responsible for protecting the infrastructure that runs all of the services offered in the Azure Cloud. This infrastructure is composed of the hardware, software, networking, and facilities that run Azure Cloud services.</p>
<p><img src="https://lh3.googleusercontent.com/6x-PZcvjmkvzOCCDTOBMHaY1Hm5vvp0wO-qpFIObr-8gb6RWNZoAb8UaZgif-oP2SRjedOXH7O9o7awgq33aaQCsbbKIZUX080XkBvv1j_0n6VyBSkNwUtVV15jzu6BPlPuVDMvJ" alt="" /></p>
<h2 id="policies-and-access">Policies and Access</h2>
<p><img src="https://lh5.googleusercontent.com/W0s8GBrFCn3oGBplWL-3CkVdELjemy_kp65eoEk8whNqvQ6wmgll3IoKc4DyPrlt9t5AeKMER9Apn-nXYvwN9kFS6EPqLsPIkU5ZPkiW-YgLQP7oLseB-1TH61ZEY0lYg3_OPUuv" alt="" /></p>
<p>In Azure everything starts with Identity and Access. All Azure resources ( Network, Compute, Apps, Data etc.) are governed by Azure Active Directory. You can also add policies and Role based access control for each individual resource in your cloud.</p>
<p><img src="https://lh6.googleusercontent.com/zFLkL5WpWCEkrfmcCLg8mOsz2oQO4XZC8MbnpFEy_CxUE4Vg4zZqFpZDnMrShnkDws_ZtafiJIpS6eP6WjZIDjudAaPWbuyDNyEF2Ab3MIsiBjLaq4yxLECQR3fjctDn7Y_7aMVF" alt="" /></p>
<p>Azure has a separate service to manage security policies and access to resources. Weather accessed by people or programs by REST API calls to your resources. This can control which processes can access your application files or data and granular access is delegated.</p>
<p><img src="https://lh6.googleusercontent.com/xyukF8A8HBqNsFmjmu6ROYDSy_SJ_2qiJWTWXFmYl9PVo2yxBs8C1HR3pd1Nm1sn5L4Nf30AL8Ul48c-rbR4J-gkXBNJgExzROrnqTIyBf3ncykLwOIJp9BcvEB09993lVa5_KEi" alt="" /></p>
<p>These controls are the front door of your environment. Your IT can make developers as contributors, marketing users as readers and so on by applying role based access control (RBAC). Role could be owner, contributor or reader and many other in-built roles are available.</p>
<p><img src="https://lh5.googleusercontent.com/zwLWOxEd1TKSFbsPziMIAJdbmc3A-I-URI21AjebsNPezKWoCRrWB79ujtZD7UkD4pXFi_S7pIpnF_v6C1Bx5u0l1o5Cg7YiKv0z_L73DWbbTWcYDUydyKuxeFFVvL3wKKzAMyqI" alt="" /></p>
<h2 id="perimeter-security">Perimeter Security</h2>
<p><img src="https://lh6.googleusercontent.com/srZpu7GCTkioKhComoj0lB1qJ2kp8rqytWL614pZplxDGpCBhHZlfbJzcL1ya5t5JQtGHvapSG8vC4bGK44fpJfwKIierON8ZIMZNh8xmQbWg4-PYi9LQTqU46R_9yGHRjRSSQwl" alt="" /></p>
<p>Perimeter networks in the cloud enable secure connectivity between your cloud networks and your on-premise or physical datacenter network. Perimeter network is also called a demilitarized zone or DMZ. The number of DDoS (Distributed Denial of Service Attacks) over 100 GB/s in <a href="https://www.comparitech.com/blog/information-security/ddos-statistics-facts/">volume increased 776 percent in Q1 2020.</a> In a DDoS attack, a perpetrator intentionally floods the system, like a server, website, or other network resource, with fake traffic.</p>
<p>In order to secure your perimeter in the cloud you must set up a Firewall. You may want to also install IPS (Intrusion Prevention Systems) & IDS (Intrusion Detection Systems ) to detect and prevent unwanted traffic and block not requiring ports and IPs.</p>
<p>Also you must protect your perimeter from Distributed denial of service (DDoS) attacks. Every property in Azure is protected by Azure’s infrastructure DDoS (Basic) Protection at no additional cost.</p>
<p><img src="https://lh6.googleusercontent.com/HWtudjFgAQZSNr0Y2MhD_vqN5VyqBWJW-gXEeDzHCSW6qop7yKcIyRIxEtQFNMJ7j4_56fOOfUpMKPcUH9BcG9Glq5O00B8LJadQQTySCywK5gZEmae1xl_wtbMTRjiX1AIiPgoc" alt="" /></p>
<p><img src="https://lh6.googleusercontent.com/2jPvLCp15Ld7pEJXg-unikeb6NU1I7jfh6dTUGuM3ouohJZIfVDljvrbuhoLSAkHsPt7D7CL4p3zHA6UNtLBt-iXKRV_XBcc-lAgv08Bpx2C6-XrJWm8Vl9WeeX3MlcV-1cdRyq4" alt="" /></p>
<p>Layer-7 DDoS protection can be achieved by using <a href="https://azure.microsoft.com/en-us/services/ddos-protection/#documentation">Azure’s DDoS Protection</a> for protection against DDoS attacks.</p>
<p>Azure DDoS protection will save from protocol attack where the attackers tries to find and exploit weaknesses in layer 3 (Network) and 4(Transport) stacks.</p>
<p><img src="https://lh4.googleusercontent.com/SgItOoTz0lqyUTFmYMGtBPiTPpzvpEggFDTAgEfS0M8rCVARiclWRimVrtCg5uqw7xYtEJDVBvZCO6fhyH7Lb92aYF5Lw2j4EmS9lFAKGI5TuWsAsgrbs5_3J0C_CMAns-NuC2kh" alt="" /></p>
<p>Application attack where the application packets are used to disrupt transmission of data between hosts like cross-site scripting or HTTP protocol violation attacks.</p>
<p><img src="https://lh5.googleusercontent.com/-TRCxfIU2zD1YXEYJ7FCUAjnPI2e1Z724VFNLNErsLzGBvE2FRtjShc8WKNUOY2k9tkbRP4m9T3OSM6KhcUI5vMt3pZalD7IemoPmdGIpbepYW-XoSAQFEwnvqQhTCTi5dUW8h4E" alt="" /></p>
<h2 id="network-protection">Network Protection</h2>
<p><img src="https://lh3.googleusercontent.com/oEyHwqk9JSt51qOeh47jEccKkAzOUgQ__XcK75rH-5qcAjvSUBUWZq2iZYgETe5oAmPH4WfaRJUwQJRu8ocIvsFmvQ1BqrB0sEHuaAjRCQp0a26zBICgb1e9MyDu5c7j_NwTfML6" alt="" /></p>
<p>Azure Security center will report on potential network and security issues related to open ports and firewall settings and Network Security Groups (NSG). You can enforce logical network boundaries and limit permissions to NSG.</p>
<p>With enabled network watchers you can quickly go to the my network diagram page and check the diagram for the selected virtual network, select the desired subnet and open/close port or block IP addresses.</p>
<p><img src="https://lh3.googleusercontent.com/b4ZBLVK9nJ5LQRfRKU8N7T9Jhu2dg_X-HDT4Hx8Q71JhVEqw8zM1pfZ0F-6vLeazZ4-M2PtSahZaezAqjm112TC2dBTtbiHEAUhyrdGO7N5ftgWrV8oeiWtbPOEamkD00v7CvoGK" alt="" /></p>
<p>If I select the backend subnet and then I can see all of the network security group (NSG) and identify the desired NSG to analyze.</p>
<p><img src="https://lh3.googleusercontent.com/QPZULg1mwiklJNjBrfkIaGKgl2m8rxJ2z4yOi9mBOa_jAuUGrmBjWLsTriGkWK0dPRTV-LZfRBhCuNR2m58bNXvny4rAxHJmx_i5Ro48g6G466De9QiYkPLcLEwBkxIhtO3xbB20" alt="" /></p>
<p>Once I get into the NSG page for a desired subnet then I can manually enable/disable ports or range of IP addresses. You can also enable just in time port access for your virtual machine and manage open ports.</p>
<p><img src="https://lh4.googleusercontent.com/ICawWtuXweQcUm3_8xxqRJ-I-Yb412SFbHSXsAQJaAMSHWI_mJwnaYAT3mnk2e_q13Ts0snemP7lgR8yyerxeUcpya-2NSgmRy4Tg8k6x54AwfbPVVYVXFS9grcA8mPYYPAXsKye" alt="" /></p>
<p>With intelligent recommendations to reduce exposure to <a href="https://en.wikipedia.org/wiki/Brute-force_attack">brute force attacks</a>.</p>
<p>Additionally, your network in the cloud can be protected by enabling monitoring, encryption and network segmentations.</p>
<h2 id="compute-protection">Compute Protection</h2>
<p>Compute means your Virtual Machines and such (IaaS) things in the cloud. Azure Security Center will also provide an actionable item list for your virtual machines. Tell you what the patches are pending with their severity using machine learning.</p>
<p><img src="https://lh6.googleusercontent.com/7-QWjkor6rRvSqmvztq3MC8gQv5zRL1jOBCVL6a2qQVLupdkkorETOSYM9FAnKa0nsDDlVZK2gx2mu6AhZHsaJVfUyjVxw0JzHWyW3xEayD-2V0MagNj8ws_BPj576AZxAV1ydkV" alt="" /></p>
<p>You should set up NSG on the Virtual machine NIC (Network Interface Card ) and secure it by disabling port or IP addresses if required. So that it will keep unwanted traffic out of the host.</p>
<p>Make sure you have installed proper antivirus in the Virtual Machines. <a href="https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/iaas-antimalware-windows">Microsoft Antimalware for Azure</a> is free real-time protection capability that helps identify and remove viruses, spyware, and other malicious software, with configurable alerts when known malicious or unwanted software attempts to install itself or run on your Azure systems.</p>
<p>Single SignOn is another feature that prevents users from entering passwords and protects us from various security attacks. Use Azure Active Directory authentication and Managed identity that includes System or User Defined Identities to auto authenticate other processes trying to access your virtual machines.</p>
<p>Virtual Patching is another workaround that we should use to protect existing legacy applications installed on cloud virtual machines. Legacy applications are not cloud ready and they have various vulnerabilities. Most of the time we do not have permission/bandwidth/budget to modify them. At that time to protect them in the cloud we can use virtual patching. Many 3 party extensions are available from azure marketplace for that. Virtual patching works with WAS and WAF together. It uses WAS( Web Application Scanning) to scan installed applications in the virtual machine and whenever it finds a threat it creates an automatic rule in the Web Application Firewall (WAF). This security option is best for many lifts and shifts of legacy virtual machines or physical machines.</p>
<p>The Azure Security center uses machine learning to continuously assess security and vulnerability levels of your Virtual Machines, Networks and service configurations. It also gives you actionable recommendations to prevent exploits before they occur.</p>
<p>Example of Virtual Machine recommendations by Azure Security Center are as below:</p>
<ul>
<li>Apply a Just-In-Time network access control</li>
<li>Enable NSG</li>
<li>Apply Disk encryption</li>
<li>Apply System updates</li>
<li>Restrict access through internet facing endpoint</li>
</ul>
<p><img src="https://lh6.googleusercontent.com/LZyrRC0vuRzAIk-eGRYkB04C4kAbHctU3BqFOHR3E43GMxVuDJosv5I2yH7r9qhkSwNWWChrAgW30bG9-jjRxsdXHtmoQc-GkBx88Fg4UJUs0aqe8JWKCyobSA2zrBXElj65-7sC" alt="" /></p>
<p>Azure Security Center has Adaptive Application Control that dynamically applies both allow and block lists to keep unwanted traffic out of your virtual machines.</p>
<p><img src="https://lh6.googleusercontent.com/jSIG8VJfAlHW8XMihUC2_-dDZJd6-yf0_YKqOS2gGFwhMViU6KskOMn16w0eaNWy1sV4TJkYZeM6T-YuPmG5TCfquu1NkrWNDCoc8O8s9k5rXzWFDhxOnMPXmSwH6HLmRhAvB9lp" alt="" /></p>
<p>If you are using Hybrid Cloud still you can leverage the Azure Security Center features for your on-premise or 3rd party cloud resources. Because Azure Security Center gives many features for your VMs in other clouds and in your data center.</p>
<p><img src="https://lh4.googleusercontent.com/w3nVZsAZMdIrNweOVxGjTDchCACkOZirqoOOo435qJZsHsChKypEJ_r0n-qSgVyweOjKCPxmQH5LGhFM1bOKyX0lWexWg8hh1nLWcAxlAxopAfvj9CM15_zg1iArsh58J6_PABVK" alt="" /></p>
<p>You can implement Intelligent Threat Detection and Response.</p>
<p><img src="https://lh4.googleusercontent.com/8ZQdTANFJN_8jsAy_u9uKs-yLy2YvfUb2UU48pUsfye2fCb3rsHU8gqJpglU7WGlFKoDt3ikdRaHar61BE3tvTJJekNxJuOvYS3bm6A-cdZFZ4A2TTXD_LPVcWjVVnia1hipbYQX" alt="" /></p>
<p>The security center leverages the Microsoft Intelligent Security Graph to discover and take action against attacks. It combines cyberintelligence that Microsoft collects across all of its services and industry data to block known attack patterns. You can also prioritize alerts and incidents that are important to you.</p>
<p><img src="https://lh4.googleusercontent.com/jvh9AeHoqP4CrjHG5yzc7rp9LoL7tQA0y5kZpEEPvNxjFsbhQX5KQ_rXOr83VNPKkrIl2GxZOC3-AFEIztNjb7boSz1J77wDUfQlzJ03lc82xot6aDUL3C1DJM3CKlBtlZRnJpws" alt="" /></p>
<p>You also get a unified view for forensics analysis and the ability to search across all of your compute resources.</p>
<p><img src="https://lh5.googleusercontent.com/8Cj9H6tZTO9V6tDB-K6kwlBLNYbyPZ3kJlHBB1tJJkzljzt1ynlasBJGPm6jciWNueT8lHRVadJ61hASdBnAfdg799S45NFFxcVOdVfIRlZTom_BXddFC_TnvryI-GAxjFZcv0Pm" alt="" /></p>
<p>You can also configure the sys log, setting the advanced threat analytics.</p>
<p><img src="https://lh6.googleusercontent.com/7z8yvlksVrU4vwYIUm_w9wSHrUOfrggbK2RXbBQJUWkXoxXc8GMTnLHBfIynzmZq2T7B_mGhZCiTgA-0IuJNx6z1g1xRL3dLEuPvhKboa-r2I36utECJFX6vjZ473O1EcaIFv_Ue" alt="" /></p>
<p>You can even visualize threat intelligence</p>
<p><img src="https://lh4.googleusercontent.com/7GV-Bl0qbpKnzUb0ncAW2wwHYUkMpYlxzNJCyHzPmyrN28xEDXZI_gWiTDeyGXP9VC7RUPVu21hiaHwo_oI73q3VaZqoMbRIcjFkZT_5DswNglQC4YLajfuP2303Pn-rND3qHYff" alt="" /></p>
<p>Threat Intelligence Log Search will show you the tracing for even most trending attack techniques and the geographic regions affected.</p>
<p><img src="https://lh4.googleusercontent.com/FsnxlZWavD8rtxz2JDVJx0Uh42Ye9YBmnpqqgopqgoa4uYc-DRqJAyp-P3E0hQWn-_kovdDPvi3Fg3Bw154cykOhtPmBsv2abPh9xNRKuoEbekSpFeuvzRFLb2bNfyTJUgpVYQrD" alt="" /></p>
<h2 id="application-protection">Application Protection</h2>
<p><img src="https://lh6.googleusercontent.com/WkCU8ENZvUKyrIsgeitfdkSoLKZ5sIIeXx-l-wG7GqdGNxfljmodVVC9tZe8nmf9TqFg-xD0_cMHwcnP-4-Xcxhk7c6WFpgRJuF1RvN6DPQ-ceesbLPA89UcYzdnuZyOVw3r0E-L" alt="" /></p>
<p>Applications are right on top of the data. Therefore, securing applications is a high priority. Maximum attackers are looking to attack applications only. We can do below things to protect our applications</p>
<ol>
<li>SSL/TLS, HTTPS</li>
<li>Single Sign On</li>
<li>Application integrity ( adhering to the policies like following NIST guidelines )</li>
<li>Vulnerability Scans</li>
</ol>
<p>Applications installed or used in Azure cloud for accessing and presenting data. Application securities are governed through data, Virtual Machines or compute (IaaS) and platform (PaaS) services in Azure. Web applications can use azure Managed Service Identities to streamline secure communications with other services in Azure connected to Azure Active Directory.</p>
<p><img src="https://lh6.googleusercontent.com/B38z9MJiPn4TGje_O2Dw1ncenCsp80skbgp9w2x9Xh_LSHFNoBAo-MUeOA5oSruenzZeS5-Mal9vCxriDJZGIlauSmh6-4dxgdZ1XY36nP6K9XR2zC6Z9EP78r4kSZ9EehsMWusm" alt="" /></p>
<p>For your applications to make your data-in-transit encrypted you must enable SSL/TLS. From Azure Web apps you can manage SSL certificates and your app can request a valid certificate for all incoming requests.</p>
<p><img src="https://lh3.googleusercontent.com/aS6swqis8f0x9QVa1p45ptE-QoOkhyB2gAdPVZJAxRjsOH4tyKRDGQEqURxloaBZfDUcBY5HOyUNyfLsKHa4cmYzH-fMZphi2gAwQWCjJp8AdoRVJEmLvbCCvBiMWCfsB_7ZbK8x" alt="" /></p>
<h2 id="data-protection">Data Protection</h2>
<p>Data protection is the most critical thing. Because it is at the core of your applications and services layer. You must secure your data at rest and in-transit. You can use encrypted volumes to get protected by underline infrastructure.</p>
<p><img src="https://lh4.googleusercontent.com/KlnifAjbBE8FlnuLghwEA6Eg8s_bE5W5azVTnF3dNbXJWO5syNrAWy_HnCu6do7O6Z_QYD3-PHqC0gs-9XMP-8FGhxkD9BlktLOPbqZjrZsx7EXfWT0qo0WHGCj19Iw09nDVg1ZP" alt="" /></p>
<p>Regardless of structured or unstructured data their protection is built-in out of the box over Azure cloud.</p>
<p><img src="https://lh5.googleusercontent.com/x1sYgv-KmQFZPKEZUzbMz74paeiGL4RRjHO2aKCTqFoeSth8x6XrirHDyq4ZiFggV8W2-E7obGfuS1WXX_L90w2zhYmyCH1k3to8WYXg5e5i9r7UId0BgaHtRgsrIbdbJwphXIe2" alt="" /></p>
<h3 id="structured-data-protection">Structured Data Protection</h3>
<p>For structured data, all data is encrypted at rest. And you can use machine learning already built-in Azure security center to proactively look for and alert you on potential security vulnerabilities.</p>
<ul>
<li>Enable Auditing & Threat detection on SQL databases</li>
<li>Enable Transparent Data encryption</li>
</ul>
<p><img src="https://lh5.googleusercontent.com/lX7hSL70qigudG3PuWcttAMffG82QDrIMo1gUyAtqDCltHBe_Uxkugyz-ozw8uTewz2NBljLhdCcojqbt4uihJTAdpUt8JsS_LzH6CukkoSrN39Q9E7W1inIeZPt6QxoKamqFoYv" alt="" /></p>
<p>In SQL server which stores structured relational data. You can enable threat detection on database level or entire server level. Threat detection can be related to data encryption, enabling in-security telemetry.</p>
<p><img src="https://lh3.googleusercontent.com/n4IyubCpQ_YTzCkULc5ZQ--Ca1hT6CQ5YGQUC3Zp7QtxwrbetwICZ5MQ2WJwsYiUIvY6f-8zAwVvi9Jn4cvJVJkC_PYAQ7ll6KBd1YPIFXi4zRy-WVnmo_6syD9lqT46cXjpuiUl" alt="" /></p>
<p>In the SQL Database service itself there is vulnerability assessment where you can get extensive capabilities to recommend and enable sensitive information discovery and classification.</p>
<p><img src="https://lh6.googleusercontent.com/n3OSbj9ATSvd62_1AjApmu08aAigVAwc0EzI4IvlT5_DJRP5B-5Kk1Yr3_7TEfdhuFRs7uQub-0-rhtuXwSA78QwImUBhMO0LP-X6_4Vb3KdvbYHaSQ72Tm1kEgg1s7DAAvfsmR8" alt="" /></p>
<p>Azure SQL database service does dynamic data masks to obscure data fields and more.</p>
<p><img src="https://lh6.googleusercontent.com/o6KpDoQ2H7o-ICfwTxu4gYAw9arevhDievkajDKCs58VCLRFzMONUQoAr05WiS2GBiLAEcufFmtL3BLB-5iKJ1tvwmJnolPZgXMb2XMIVaBGEFRux4THygOGA-pyATxp7xDDShLd" alt="" /></p>
<p>Database service just needs to enable these threat protection then Azure security Center will alert you for any vulnerabilities found.</p>
<p><img src="https://lh4.googleusercontent.com/YwiFtzKgXgP04_k09mqovNccFzWdC42wT-xNfMzweS5sb5AQ3WQhy7FTEOqhJtECvpzM4fO-HXyUUGdrqC7zTJgFa4V7gEMFwrE-nzb1rhsUgwEUrOZd1uuTAgV819eiYhw3aBN_" alt="" /></p>
<h3 id="non-structured-data-protection">Non Structured Data Protection</h3>
<p>Non structured data like blobs, files, tables and queues are also encrypted at rest in azure cloud and each account is geo-redundant.</p>
<p><img src="https://lh4.googleusercontent.com/QuAOQ4RtCAIH3eZ5-WivrHKulS2iAe20rsKlwa5_5WtoXClu2B8LQ4nk3jtLaCQJIXVjOUIJdreH8lcEHbyeiAPqVr8EcPYD6hbRte4gColF7rIDbIVgQsBX2vJWa0jZbqzIQCTE" alt="" /></p>
<p>You can use access keys to control authentication, Shared access signature for secure delegate access and granular firewall controls to restrict public network access.</p>
<p><img src="https://lh6.googleusercontent.com/_K2uqutTalHQG0iCyLRzNkXZCUF9MtBpexeYRLw8TBX2i64QaYiTrYqstm_M9tg6soeZDeKsczQyZVWoSunhUAOaGb3QHa3prbwiW4-vZISyuIupnsjnw6wjKxRkdIOSvUUnjVIq" alt="" /></p>
<p>Azure Security Center will report its findings whenever security at risk or protections are disabled by your admin.</p>
<p><img src="https://lh5.googleusercontent.com/vDS4A324c7pmIBqMj5KofsEpSeyb9RtnsRk7XXcOjIv-fK_NUhkV2zVFUQ4wjYytTqRHEEw8zWqcsasTvl-ETP1IrJ64msos0R3f_3HK3udHeHu3WMl1fnLwRsXBWjrPHw8DT89J" alt="" /></p>
<h2 id="summary">Summary</h2>
<p>The idea behind the defense in depth approach is to defend a system against any particular attack using several independent methods. Defense in depth is in-built in Azure cloud and it facilitates keeping the security in multiple layers of your architecture. I am also sure the same kind of facilities are available in other cloud environments like in Amazon AWS & Google Cloud. So it’s a matter of awareness and making your cloud secure by properly utilizing the services given by you cloud provider.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariIn a traditional data center you create one perimeter, secure them by installing firewall, WAF, SIEMS etc. and have 100% confidence that the data center is secure. However, when you migrate to cloud then your VPC or Virtual Network is not 100% secure you have to not only secure your perimeter however, inside of perimeter you have to take care of security. Since it’s a public cloud you have to be more cautious and make sure whatever workload or source code you are moving to is cloud aware. There are various best practices at each level you have to take care of in order to secure your cloud.Cost Optimization with Amazon EC2 Spot Instances2021-12-11T00:00:00+00:002021-12-11T00:00:00+00:00https://www.rupeshtiwari.com/cost-optimization-with-aws-spot-ec2-instances<blockquote>
<p>Creating cost optimized highly scalable cloud architecture is challenging but it is possible! Using Spot Amazon Elastic Compute Cloud (Amazon EC2) instances you can save up to 90% of on-demand pricing. In this article you will learn what spot instances and what should be your strategy to use spot instances. Additionally, I will give you scenarios where most of the customers are using spot instances.</p>
</blockquote>
<h2 id="amazon-ec2-purchasing-options">Amazon EC2 Purchasing Options</h2>
<p>Amazon Elastic Compute Cloud (Amazon EC2) offers 3 purchasing options:</p>
<ol>
<li>On-Demand</li>
<li>Reserved Instances</li>
<li>Saving Plans</li>
<li>Spot Instances</li>
</ol>
<p>All of the above Amazon EC2 purchase options are also available in Microsoft Azure and Google Cloud Platform (GCP). So if you are still in a Multi-Cloud environment still you must continue reading this article.</p>
<p>Normally, when you are designing your workload in the cloud. Make sure you combine all three of them to get better performance and huge cost savings. Amazon Auto Scaling Group (ASG) can help you to set up rules and conditions to add the desired type of the instances when set up correctly. You can combine on-demand, reserved and spot instances in a single ASG.</p>
<h3 id="on-demand-instances">On-Demand Instances</h3>
<p>Pay for compute capacity by the second or hour with no long-term commitments. Mostly you will use these instances for the spiky workloads or to define needs of your workload. Example: Netflix uses on-demand EC2 instances at off hours to split, encode the video files.</p>
<p><img src="https://i.imgur.com/BAd22sC.png" alt="" class="full" /></p>
<h3 id="reserved-instances">Reserved Instances</h3>
<p>Significant discount (up to 70%) compared to on-demand instance pricing. You should use these types of instances for steady state applications or predictable usages, databases. <img src="https://lh6.googleusercontent.com/IYJsC3v5updKOqxDw2n2EdrOW6aG6tAU_D4iS0E-WgwCe0zsOK98aar8fYibwrKCu4CYfszSTxQyhqdjksyYIr14hq4Xgb1fJhXTpVk8IzHRu69ok4uJ19xNh86dWPZ1oAOegL9V" alt="" /></p>
<p>Example: your production servers or database servers or domain controllers that work regularly every year. You don’t want to turn them off for several years and keep these instances for the next 3-4 years or more. Then you can purchase reserved instances, with discounted prices and save huge costs.</p>
<h3 id="saving-plans">Saving Plans</h3>
<p>Same level of discount as you get from Reserved Instances however, you get more flexibility.</p>
<p>The flexibility to access computers across EC2 and AWS Fargate.</p>
<p><img src="https://lh6.googleusercontent.com/e-U_KjFJXnLZkbMpwMNWAd-v4rOxSAdTDg8fDOMxSTGgHfM86RMQoJ7_jjATpQ8BhfhtsK5A1Axx-6VUH56CAyRDJj6o0QTE_IwMUoikFg8bezlCc-fAzZ2f_8nx2cj0vGQ4JMtA" alt="" /></p>
<h2 id="spot-instances">Spot Instances </h2>
<p>Spare Amazon EC2 capacity for up to 90% discount compared to on-demand instance pricing. You can use these instances for fault tolerant stateless applications, instance flexible or time-sensitive workloads. Spot instances are identical to on-demand instances in terms of performance, infrastructure platform.</p>
<p><img src="https://lh3.googleusercontent.com/61yRh1SvEhkZ2ZMwHTWxlMFWE2T-JOxMax217ZHg6uaUgIRF8FKkpDVtV0SJ1bbPXE2kTAJlwOfwDSzqIQOzOrcnrfwZhK_tfdeoB6dprd1v7d-wDJ4nzs8YenGlPAYk8v8TQUvG" alt="" /></p>
<p>One thing to note is that AWS can reclaim these instances with just a 2 minute notice. Therefore, you must plan for a backup strategy while you are using spot instances. Or try to use the spot instances for the use case where you do not want to store the files, data, state or session. You should do stateless activities on spot instances to be on safer side. Amazon EC2 Spot instances are low cost, easy to use and flexible. In Big Data and Analysis use cases lots of customers are using spot instances.</p>
<h2 id="aws-spare-capacity">AWS Spare Capacity</h2>
<p>Amazon AWS does not reserve a minimum or maximum amount of capacity in spare. However, for the on-demand instances in order to give elasticity AWS has to keep spare capacity. Additionally, when a customer is not using on-demand instances then those capacity AWS keep it for spot customers to leverage the available capacity. AWS offers a huge up to 90% discount to spot customers with the caveat that whenever AWS needs the capacity back from you for on-demand customers, AWS will give you a 2-minute warning and take the server back off from you.</p>
<p>On average, every week, AWS customers are using more compute capacity on Amazon EC2 Spot Instances than customers in 2013 were running across all of Amazon EC2.</p>
<h2 id="spot-instance-use-cases">Spot Instance Use Cases</h2>
<p>Spot instances are perfectly good for below scenarios:</p>
<ul>
<li>Fault-tolerant</li>
<li>Flexible workloads</li>
<li>Loosely coupled systems</li>
<li>Stateless workloads</li>
</ul>
<p>In all of these above scenarios, you can trade off spot instance interruption with the huge money saving. Many customers are using spot instances for below use cases:</p>
<ul>
<li>Big Data</li>
<li>CI/CD</li>
<li>Web Services</li>
<li>HPC (High Performance Compute)</li>
</ul>
<p>Big Data use cases you might have some state while computing and when you lose the computer with 2-minute notice then your work is interrupted. However, in order to not lose state make sure you are using non-instance state like s3, HDFS, etc.</p>
<p>CI/CD use cases you can use spot instances. In CI/CD server you run build, test etc. Maximum time CI/CD tasks, workflows and pipelines are well capable to handle any kind of server failure. Therefore, a good use case for spot instances.</p>
<p>WebServices can use spot instances. If you are designing stateless and scalable web services by creating multiple spot instances putting them behind the load balancer then in-case one machine is taken away by AWS, you have other capacity to server for future requests. Only 5% of the time your one instance will be interrupted by AWS so existing running work interruption chances are there but very less. Since most of the time your work will be finished.</p>
<p>High Performance Computers (HPC) can use spot instances. In the Finance domain most of the Grid that uses HPC with high throughput computing workloads which is loosely coupled can be done using spot instances. In Grid single node failing does not destroy the entire cluster. Similarly for big data we can use spot instances where anytime interruption occurs it’s just a matter of quickly reprocessing.</p>
<h2 id="spot-market">Spot Market</h2>
<p>The Spot Market is the unique combination of instance type and availability zone. The supply and demand of the instance type in that availability zone is what drives the price.</p>
<p>The following screenshot shows a price comparison for different instance types. If you see the price history of c5 large instance types over the past 3 months it is pretty stable. In this graph below we are looking at the spot market for 4 different type of instances.</p>
<p><img src="https://lh5.googleusercontent.com/q_tMVbNI0yZn2rtOPE10dc1DmzStsmm_TrGO-wknKHSyj6R_4WVaRyTq8GVhf3m74_Dpl_hH5Cz9CBKB_D3WnE40z3KL0fXf-P_fLvd7ar6BO7xIMnuqSzQJGNAYkPXfnk5Xs2QD" alt="" /></p>
<h2 id="spot-instance-best-practices">Spot Instance Best Practices</h2>
<p>While AWS never runs out of spot capacity it is possible for a specific instance, for a specific zone might not be enough spot instances available for you. Therefore, if you ask for 100 spots you may get only 50 of them. So in order to solve this problem you should not be tied with one type of spot instance rather you should use the spot capacity pool given by AWS.</p>
<p>Additionally you must plan for interruptions since at 2 minute notice AWS can take back the spot instance from you. Over the last 3 months, 95% of spot instance interruptions were from a customer manually terminating the instance because the application had completed its work. So basically 5% of the time Amazon actually interrupts customers so 95% of time you should be able to work with spot instances without interruption.</p>
<p>If you are architecting a highly scalable workload then Amazon auto scaling group (ASG) can help you to select the instance type, spot allocation strategy like cheapest instance, also define the split percentage between spot and on-demand instances and let ASG handle this AWS spot instance take back interruption. AWS will automatically spin up new spot instances. I have seen customers define base on-demand instances and in order to boost their application they utilize spot instances and save on 90% cost.</p>
<p>Either you can set up ASG to spin 100% spot instances on top of the on-demand instances as your base capacity.</p>
<p><img src="https://lh6.googleusercontent.com/vLP5Kp8O44m7WC6gyh3R4LYLsY__8aKucZS0Gdmjmu2q6zhh3wf0s5Qi7U7QNfFUq4SOL_R12267Z4iZsFZHd1HJVGlLx0cVGMNi0A3GLk_j5c8dl7g9iCUePhW5_ME44JnP75CQ" alt="" /></p>
<p>Many customers also set up ASG to spin 50% spot instances and 50% on-demand instances on top of spot instances. Still keep their base capacity as on-demand instances.</p>
<p><img src="https://lh5.googleusercontent.com/GVQA8D6imd0PjyIGv3VVKM6NwH-G79FWS5q-6M7JervqfwXT2R_I5twCH-qs2HsGMvZtswLABXnM-IYObSuW40FrAYpldcRXZLCRTAm1jXskooOg8A9Lhkx6lCxUFrqkHAes6cS6" alt="" /></p>
<p>Amazon EC2 Auto Scaling dynamically reacts to the changing demand and optimizes cost as well. ASG has Fleet management where it replaces unhealthy instances. ASG also has Dynamic scaling to scale on demand.</p>
<p>In order to optimize your Amazon EC2 workload combine the purchase options.</p>
<ul>
<li>In order to run steady-state workloads use reserved instances or saving plans,</li>
<li>For bursty, new or stateful spiky workloads use on-demand instances and</li>
<li>For fault tolerant applications, flexible and stateless workloads use spot instances.</li>
</ul>
<p><img src="https://lh6.googleusercontent.com/PYRzFV0AubudQEFWGltkzw2F8ty_VISvhdUa_ast4DyjOHCssfKzgw1f2PmM6SBm3L04K2bOxAHGYiTk59wd0xgpi4eTgWELvy3tyfD9itGrpXVduqvvpmraspMkyBD8D8k8QZ5i" alt="" /></p>
<h2 id="bursty-workload-optimization-strategy-with-spot-instances">Bursty Workload Optimization strategy with Spot Instances </h2>
<h3 id="bursty-workload-example">Bursty Workload example</h3>
<p>In typical capital market domain different types of analytical workloads are there such as below:</p>
<ol>
<li>Research
<ul>
<li>How to increase profit or reduce risk of a portfolio</li>
</ul>
</li>
<li>Reactionary
<ul>
<li>Human or an automatic system determine significant market volatility in the middle of trading day</li>
</ul>
</li>
<li>Overnight Jobs
<ul>
<li>At the end of the trading day, you collect most recent input for your models and you run them at large scale</li>
</ul>
</li>
<li>Model backtesting
<ul>
<li>When you make code changes to the model you need to test the changes with 10-20 years of historical data. High resource intensive workload and you need performance since you want to deploy the model in production as well.</li>
</ul>
</li>
</ol>
<p>In order to support such a high bursty workload you need infrastructure to auto spinning spot instances and shut them down once a task is done. You can use your own job scheduler or open source from HashiCorp as below.</p>
<ul>
<li><a href="https://www.nomadproject.io/docs/internals/scheduling/scheduling">Nomad</a> : Job Scheduling</li>
<li><a href="https://www.consul.io/">Consul</a> : Health Check, Load Balancing and Service Discovery</li>
</ul>
<p>Let’s take one high bursty workload example:</p>
<p>Trading desk running on 2000 core scale to 15,000 core scale during trading hours and overnight it runs about 30,000 core scale. This is the bursty nature of the system in action.</p>
<p>User sends the work job to Queue, the auto scaler docker container checks the state of capacity and talks to the provisioning system and spins up a couple of new capacities. Once the queue is drained it submits the request to shut down the extra capacity to the provisioning system.</p>
<p>2000 jobs are put on the queue at the beginning of trading day they drain quickly. Then 8000, 15000 jobs are put on the queue they drain quickly.</p>
<p>Similarly at the beginning in order to handle 2000 jobs you need 1000 workers then they all shut down again this is very bursty nature. Similarly at different time spans of the day different sets of workers will spin up like 3000 workers for 2 hours, next 4000-7000 workers for next 2 hours and so on. These are all bursty behaviour of your system.</p>
<h3 id="spot-instances-strategy-for-bursty-workloads">Spot Instances Strategy for Bursty Workloads</h3>
<p>How would you strategize your spot market such that you create a healthy and performant workload in this situation?</p>
<p><strong>Diversified allocation strategy</strong> across as many instance pools as possible is the key to quickly achieving scale and minimizing the size of termination events.</p>
<ul>
<li>Configure your fleet to start up in multiple spot markets in parallel</li>
<li>Build your Launch spec a large number of spot markets that you need to accept resources</li>
<li>It will give you quick fleet whenever you need to compute in high volume</li>
<li>Your instances are spread out across a large number of spot markets with few instances belonging to a single spot market.</li>
<li>So if you get termination notification from AWS in one spot market it wont impact much in your over workload.</li>
</ul>
<p>In order to spin up 32 instances each of 30,000 core clusters of spot instances may take an hour or so. Therefore, if you use multiple spot market pools in your launch specification that can reduce your fleet startup time to around 15 minutes. Your startup time can be sped up 70-80% with this diversified allocation strategy. So create a pool of instances and diversify them.</p>
<p><strong>Make it as simple</strong> as possible. AWS gives variety of EC2 instances you can group machines by their features such as</p>
<ul>
<li><strong>Compute intensive</strong>: high compute capacity category</li>
<li><strong>Memory intensive</strong>: memory optimized capacity category</li>
<li><strong>Standard</strong>: rest machines can be categorized as standard</li>
</ul>
<p>This way you can abstract out the AWS terminologies, instance types, number of cores etc. Rather you define workload memory intensive if they require more memory during work.</p>
<p><strong>Build in safeties governors</strong> everywhere. Tag your resources such that you know which team stakeholders are responsible for the payment. Put a threshold on capacity by applying some policy and governance to not spin 3000 vCore machines if you need only 1000 vCore. If you made a mistake and scale up unnecessary then you will end up paying a couple of thousands extra. Therefore, you have to apply strong governance and safety.</p>
<p><strong>Monitor & Alert</strong>: If you have bursty large-scale systems where 30,000 vcore spot fleets and 3 or 4 of them running. However, if your system is not working then you are wasting your huge amount of money. Therefore, you must set up monitoring and alerts to send sms to the core team to get human involvement to tackle the next set of actions for a critical incident.</p>
<h2 id="summary">Summary</h2>
<p>The Amazon bill is pretty complex. I am pretty sure every cloud provider bills must be the same structure. You have a lot of things and various associated costs with your virtual machines / EC2 instances. Therefore you must do your own maths and explore your cost in depth. There are many other costs you have to consider, not just the machines. Cost management is not a one day job. You must monitor them very frequently. You can check quarterly and also check future expected bills etc to learn more and tune them. Spot instances are great but power comes with responsibility. So make sure the applications that you are designing to run on spot are fault tolerant, stateless and scalable.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariCreating cost optimized highly scalable cloud architecture is challenging but it is possible! Using Spot Amazon Elastic Compute Cloud (Amazon EC2) instances you can save up to 90% of on-demand pricing. In this article you will learn what spot instances and what should be your strategy to use spot instances. Additionally, I will give you scenarios where most of the customers are using spot instances.How to Maintain Well-Architected-Framework on Cloud2021-12-10T00:00:00+00:002021-12-10T00:00:00+00:00https://www.rupeshtiwari.com/how-to-maintain-well-architected-framework-on-cloud<p>How to make sure our workload in the cloud is secure, reliable. How to monitor the cost of your cloud resources consumption? Is there any automation you can do to remind you of your cloud budget expenses, future expected cost? Is there any on-click quick fix to terminate underutilized virtual machines. Well the answer to all of these questions is <strong>Microsoft Azure Advisor</strong>. Yes, you guessed it I will talk about azure advisor in this article. This is an important topic so stay tuned.</p>
<h2 id="azure-well-architected-framework--review">Azure Well-Architected Framework & Review</h2>
<p><a href="https://docs.microsoft.com/en-gb/assessments/?mode=pre-assessment&session=local">Azure well-architected framework</a> provides architectural best practices across the five pillars for designing and operating systems in the cloud. Those 5 pillars are as follows:</p>
<ul>
<li>Operational Excellence</li>
<li>Performance</li>
<li>Cost</li>
<li>Reliable</li>
<li>Secure</li>
</ul>
<p>Microsoft has a <a href="https://docs.microsoft.com/en-gb/assessments/?mode=questionnaire&session=d5ecfe19-a58f-497d-9df4-0b176437c610&question=threat&category=Security">well architected review</a> that will give you recommendations that customers can focus on. It will include existing workload and recommend things to learn and implement to improve.
<img src="https://i.imgur.com/xYMTMmS.png" alt="" class="full" /></p>
<h2 id="azure-advisor">Azure Advisor</h2>
<p>Azure advisor gives recommendations to all cloud resources based on a well architected framework. You can access Azure Advisor recommendations as an owner, contributor, or reader of a subscription. Once moving workload on the cloud. Azure advisor is your personal advisor to keep you workload standout on 5 pillars of a well architected framework.</p>
<p><img src="https://i.imgur.com/kCqwu9D.png" alt="" class="full" /></p>
<h2 id="overall-advisor-score-for-c-level-executives">Overall Advisor Score for C-Level Executives</h2>
<p>Azure Advisor shows overall score this is the high level view for your entire resources present within the subscription. The higher the score the healthier your technical portfolio is. This is the number that is more likely you will show to your CTO.</p>
<p><img src="https://i.imgur.com/4ktlepW.png" alt="" class="full" /></p>
<h2 id="creating-advisory-automatic-alerts">Creating Advisory Automatic Alerts</h2>
<p>Checking these recommendations given by Azure Advisor manually is not a great idea. Therefore, you must create automatic alerts in Azure Advisor. These alerts can do various actions including sending sms, emails to core team or even calling webhook to create tickets in zendesk.</p>
<p><img src="https://i.imgur.com/HKw9O7g.png" alt="" class="full" /></p>
<p>You can even create a digest for a given subscription and select the desired 5 pillars and weekly you will get a summary of Azure advisor recommendations.</p>
<p><img src="https://i.imgur.com/pJWFzP2.png" alt="" class="full" /></p>
<h2 id="how-do-you-know-advisor-score-is-good">How do you know Advisor score is good?</h2>
<p>You can check the score history by monthly/weekly or days and understand if the score is increasing or decreasing by benchmarking it.</p>
<p><img src="https://i.imgur.com/0vGPqKw.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/Y7KjbnJ.png" alt="" class="full" /></p>
<p>The best way to increase score is by:</p>
<ul>
<li>Split your workload that is business critical and make it more reliable by following all recommendations and tradeoff on cost.</li>
<li>Split your workload by environment for production/non-production subscriptions. For a development/test environment, cut your cost and trade off on reliability.</li>
</ul>
<h2 id="how-to-improve-cloud-advisor-score">How to improve Cloud Advisor Score?</h2>
<p>You must go to the Azure Advisor and check all of the 5 tabs related to the 5 pillars of a well architected framework and follow the quick fix and recommendations to improve your score.</p>
<p><img src="https://i.imgur.com/WBYLnMb.png" alt="" class="full" /></p>
<p>Here are the proposed recommendations that you should follow in order to make your system adhere to a well architected framework.</p>
<h3 id="operational-excellence">Operational Excellence</h3>
<p>Operational excellence recommendations in Azure Advisor can help you with:</p>
<ul>
<li>Process and workflow efficiency.</li>
<li>Resource manageability.</li>
<li>Deployment best practices.</li>
</ul>
<p>Below are the recommendations that you can see in the Operational Excellence tab of the Advisor dashboard.</p>
<ul>
<li>Azure Policy recommendations like add tags, restrict creating resources to certain regions etc.</li>
<li>Design your storage account to prevent reaching the maximum subscription limit.</li>
<li>Enable traffic analytics to view insights into traffic patterns across azure resources.</li>
<li>Increase vCPU limits for your deployments for Pay-As-You-Go subscription.</li>
</ul>
<h3 id="performance">Performance</h3>
<p>The performance recommendations in Azure Advisor can help improve the speed and responsiveness of your business-critical applications.</p>
<ul>
<li>Reduce DNS TTL (time-to-live) on your traffic manager profile to fail over to healthy endpoints faster.</li>
<li>Improve database performance based on usage history</li>
<li>Upgrade library to latest versions for better reliability and performance example storage client library version.</li>
<li>Use managed disks to prevent disk I/O throttling</li>
<li>Improved VM performance and reliability using premium storage that gives SSD I/o-intensive disk for low-latency high IOPS.</li>
<li>Improve MySQL connection management by reducing the number of short-lived connections and eliminating unnecessary idle connections.</li>
<li>Use ARM template for template deployment, security</li>
</ul>
<h3 id="cost">Cost</h3>
<p>Here are some examples of the recommendations given by Azure advisor to save your cloud cost.</p>
<table>
<tr>
<td><strong>Potential Yearly Savings</strong>
</td>
<td><strong>What to do?</strong>
</td>
<td><strong>Impacted Resources</strong>
</td>
</tr>
<tr>
<td>1864 USD
</td>
<td>Right Size or Shutdown underutilized virtual machines. CPU utilization < 5%
</td>
<td>2 Virtual Machine
</td>
</tr>
<tr>
<td>80 USD
</td>
<td>Delete Public IP address not associated to a running Azure resource
</td>
<td>2 Public IP Address
</td>
</tr>
</table>
<h3 id="reliability--high-availability-recommendations">Reliability / High Availability Recommendations</h3>
<table>
<tr>
<td><strong>Potential Benefits</strong>
</td>
<td><strong>What to do?</strong>
</td>
<td><strong>Impacted Resources</strong>
</td>
</tr>
<tr>
<td><strong>Ensure Business Continuity</strong> through VM resilience
</td>
<td>Add more VM for improved Fault Tolerant
</td>
<td>Availability Set
</td>
</tr>
<tr>
<td>Improved <strong>Data Resilience and Performance</strong>
</td>
<td>Enable VM backup to protect your data from corruption and accidental deletion
</td>
<td>Virtual Machine
</td>
</tr>
<tr>
<td><strong>Ensure Business Continuity</strong>
</td>
<td>Use Multiple AZ for data center level disaster
</td>
<td>Availability Zone
</td>
</tr>
<tr>
<td><strong>Save and Recover </strong>your data when blobs or blob snapshots are accidentally overwritten or deleted
</td>
<td>Enable soft delete to protect your blob data
</td>
<td>Storage Account
</td>
</tr>
</table>
<h3 id="security">Security</h3>
<table>
<tr>
<td><strong>Potential Benefits</strong>
</td>
<td><strong>What to do?</strong>
</td>
<td><strong>Impacted Resources</strong>
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Web apps should request an SSL certificate for all incoming requests
</td>
<td>8 App Services
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Private endpoint should be configured for key vault
</td>
<td>3 Key Vaults
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Azure Cosmos DB accounts should have firewall rules
</td>
<td>1 Cosmos DB account
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Storage account public access should be disallowed
</td>
<td>2 Storage Accounts
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Storage accounts should restrict network access using Virtual Network Rules
</td>
<td>2 Storage Accounts
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Virtual Network should be protected by Azure Firewall
</td>
<td>1 Virtual Network
</td>
</tr>
<tr>
<td><strong>Prevent </strong>potential security breaches
</td>
<td>Diagnostic logs in Search Services should be enabled
</td>
<td>1 Search Service
</td>
</tr>
</table>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariHow to make sure our workload in the cloud is secure, reliable. How to monitor the cost of your cloud resources consumption? Is there any automation you can do to remind you of your cloud budget expenses, future expected cost? Is there any on-click quick fix to terminate underutilized virtual machines. Well the answer to all of these questions is Microsoft Azure Advisor. Yes, you guessed it I will talk about azure advisor in this article. This is an important topic so stay tuned.Introduction to CAP Theorem2021-12-09T00:00:00+00:002021-12-09T00:00:00+00:00https://www.rupeshtiwari.com/introduction-to-cap-theorem<blockquote>
<p>Are you planning to design your next advanced distributed architecture? Make sure you are aware of the concepts such as high availability, consistency and partition tolerance. The CAP theorem states that a distributed system cannot simultaneously be consistent, available, and partition tolerant. CAP Theorem is very important in the Distributed systems, Microservices architecture, Big Data world, especially when we need to make trade offs between the three, based on our unique customer’s use case.</p>
</blockquote>
<p>CAP stands for C - Consistency, A - Availability, P - Partition tolerance.</p>
<p>While building a distributed system you will get 2 things up and running at a time and the 3rd thing you will have to sacrifice. CAP theorem helps system architects to make better decisions to trade off to design distributed systems based on requirement and customer priorities.</p>
<p>AP - If you make the system Highly Available and Partition Tolerance then you will not get immediate consistency. It has to be eventually consistent.</p>
<p>CA - Consistent and Highly available system will be not partition tolerant at all times</p>
<p>CP - Consistent and Partition Tolerant system are not highly available</p>
<p>Even though you try to make all 3 features to be available in a distributed system you will have to trade off on any one feature.
<img src="https://i.imgur.com/5ApKUNW.png" alt="" /></p>
<h2 id="availability--partition-over-consistency-example">Availability & Partition over Consistency Example </h2>
<p>The application Twitter is highly available and has good partition tolerance. System has to be shared in a very smart way. So it has to be highly available. Here you can sacrifice Availability. It doesn’t matter if I get the tweet from someone immediately. It’s okay to get after 5 min to receive notifications. However, you should be available and partition tolerant</p>
<h2 id="availability--consistency-over-partition-tolerance-example">Availability & Consistency over Partition Tolerance Example </h2>
<p>Payment service you can not comproosed with consistency where you withdraw the amount and the bank backend when they query and they see a different amount is not acceptable. You can compromise with partition tolerance however, the payment system has to be highly available.</p>
<h2 id="what-is-consistency">What is consistency?</h2>
<p>Consistency in a distributed system is regardless of the node you pick, data you will get for a query will always be the same. Either you get the latest information or you do not get any information.</p>
<p>If service 1 updated N1 and service 2 query N2 it should get the latest info updated by service 1 at N2.
<img src="https://i.imgur.com/wLesuWU.png" alt="" /></p>
<h2 id="what-is-availability">What is availability?</h2>
<p>Every request that comes to the data store should get a response and it should not timeout. It can fail for authentication authorization but not for timeout.</p>
<p>If you have 4 nodes out of that N3 went down. Any application that is connected should not be directed to N3 and routed to a healthy node. So that your system is highly available. In case your request goes to N3 and the application shows 500 server errors your system is not highly Available. Your system should automatically route the request to the healthy available nodes hence make your system highly available. <img src="https://lh5.googleusercontent.com/s-Vx-ep2gasLiH_MSzV8muqDD5t6t-3SqUrULQ-kjQGNFQb8Z9NGGWNsnB2IIMvwmi-wlXnp-kos2ajGdfx5IAZZ_V3cp6z9D7_ynkE_VxILn091s0xXITMo1ui3jQfmdGyVKsj8" alt="" /></p>
<h2 id="what-is-partition-tolerance">What is Partition Tolerance? </h2>
<p>Even though the connection between some nodes break, still every request coming to your server should serve and respond.</p>
<p><img src="https://i.imgur.com/ZHFtnBf.png" alt="" /></p>
<p>Suppose service updated the user address in N3 and to make the system partition tolerant the new data has to be replicated in all of the nodes. If N4 is disconnected or down so if the same service query the updated latest info of the user from N4 it will not get the latest information. In this scenario your application is still available and partition tolerant however it is not consistent. So whenever the connection is resumed then N4 will be updated.</p>
<h2 id="can-we-achieve-all-3-features-in-a-distributed-system">Can we achieve all 3 features in a distributed system?</h2>
<h3 id="scenario-where-you-want-availability-and-partition-tolerance">Scenario where you want Availability and Partition Tolerance</h3>
<p>At a given moment a system can’t be consistent if it is highly available and partition tolerant.</p>
<p>Suppose if you have 4 nodes. One node gets updated with the latest data. In order to keep our system partition tolerant we have to replicate the data in all of the nodes. In the case of a larger system there could be 100, 000 of nodes and while it is replicating data. If someone query to the node where replication is pending at that time your system will return stale data and is not consistent.</p>
<p>Eventually all of the nodes will be synced up and you will get the latest info that is called eventual consistency. However, there is a certain period of time when the system will not be consistent. In Twitter You must tradeoff consistency however, you must invest on availability and partition tolerance.</p>
<h3 id="scenario-where-you-want-availability--consistency-as-high-priority">Scenario where you want Availability & Consistency as high priority</h3>
<p>For RDBMS suppose you have a Master Slave kind of scenario where you read and write in the primary instance of the database and continuous replication is happening in the secondary database you can use Amazon AWS RDS service. You will have consistency and availability. You do not have partition tolerance. Since you are not sharing your data. When you are making your system like payment, security situation.</p>
<h3 id="scenario-where-you-want-consistent-and-partition-tolerant">Scenario where you want Consistent and Partition Tolerant </h3>
<p>Suppose you make changes to N1 and update user info. IF you want to keep it consistent and partition tolerant to then in order for a service to read the consistent data from N3. You have to wait for sometime to allow your data to be replicated fully to N3 till then almost your system is unavailable so in order for it to be consistent your system has to be unavailable means sacrificing the availability part of it.</p>
<h2 id="summary">Summary</h2>
<p>For the brief period of time you are losing either Consistency, Availability and Partition tolerance. That is how eventual consistency is now popular: you are making a highly available and partition tolerant system and make your system consistent eventually. After knowing the pillars of CAP theorem you will be now able to design distributed systems exactly by knowing your customer requirements and concerns.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariAre you planning to design your next advanced distributed architecture? Make sure you are aware of the concepts such as high availability, consistency and partition tolerance. The CAP theorem states that a distributed system cannot simultaneously be consistent, available, and partition tolerant. CAP Theorem is very important in the Distributed systems, Microservices architecture, Big Data world, especially when we need to make trade offs between the three, based on our unique customer’s use case.Introduction to Point of Presence and Network Access Point2021-12-09T00:00:00+00:002021-12-09T00:00:00+00:00https://www.rupeshtiwari.com/introduction-to-point-of-presence-and-network-access-point<blockquote>
<p>Point of presence is a fundamental block of the telecommunication domain. You must be aware of this concept. If you want to system design for video content delivery like YouTube, Netflix or Hotstar.com then you must learn what is PoP and how this helps to propagate massive amounts of content across the globe with faster performance and lower latency.</p>
</blockquote>
<h2 id="what-is-the-internet">What is the internet?</h2>
<p>The internet is a massive network of interconnected smaller networks.</p>
<h2 id="what-is-the-point-of-presence-pop">What is the Point of Presence (PoP)?</h2>
<p>Point of Presence (PoP) is the Network Interface Point between communicating entities. A PoP is located in a data-center that houses servers, routers, network, switches, multiplexers and other network interface equipment. PoPs are located at colocation centers and Internet exchange points. A PoP is the local access point for an Internet Service Provider (ISP). PoP consists of high-speed telecommunications equipment that helps customers to connect to the internet via their ISPs.</p>
<p><img src="https://i.imgur.com/E4bOvEl.png" alt="" class="full" /></p>
<p>(image taken from <a href="https://networkencyclopedia.com/point-of-presence-pop/">https://networkencyclopedia.com/point-of-presence-pop/</a> )</p>
<h2 id="point-of-presence-in-action-example">Point of Presence in Action example </h2>
<p>Chris is watching Sky News on his laptop, Chris Internet Service Provider (ISP) is Verizon Fios internet service provider. So Chris is part of Verizon Internet’s large network. When Cris types <a href="http://www.skynews.com">www.skynews.com</a> his request first goes to Verzon’s Point of Presence. But the sky news site is on another network. It is on skynews web servers.</p>
<h2 id="what-is-network-access-point-nap">What is Network Access Point (NAP)?</h2>
<p>So we have 2 different networks. Each one has a point of presence. But how does Chris’s request get on to the Sky’s Network?</p>
<p>We need a Network Access Point (NAP) , a NAP is a building where Internet Service Providers can switch information on to each other’s network. Now Chris’s request can get all the way over to skynews.com by using the NAP. And skynews.com can get all the way back to Chris’s computer.</p>
<p><img src="https://i.imgur.com/v94nvmL.png" alt="" class="full" /></p>
<p>And there are thousands of PoPs and NAPs all around the world, helping us all stay connected.</p>
<p><img src="https://i.imgur.com/WkeiuwV.png" alt="" class="full" />
(image taken from <a href="https://www.youtube.com/watch?v=vUD8U7eLbQ0&ab_channel=BrooksBankict">https://www.youtube.com/watch?v=vUD8U7eLbQ0&ab_channel=BrooksBankict</a>)</p>
<h2 id="what-is-a-colocation-center">What is a Colocation Center? </h2>
<p>Colocation Center is a type of data center also called a “Carrier Hotel”. Colocation facilities provide equipment, space, bandwidth for rental to retail customers. Colocation facilities also provide space, power, cooling and physical security for the server, storage and networking equipment to other companies.</p>
<h2 id="summary">Summary</h2>
<p>So when you are social networking, shopping or surfing the web, you are using lots of interconnected, smaller networks. AWS CloudFront has more than 200 PoP across the globe helping customers to deliver content faster with lower latency and high performance. In AWS PoP also provides DDoS Protection, Attack Mitigation and certificates management features. Amazon prime videos are delivered via AWS PoP only.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariPoint of presence is a fundamental block of the telecommunication domain. You must be aware of this concept. If you want to system design for video content delivery like YouTube, Netflix or Hotstar.com then you must learn what is PoP and how this helps to propagate massive amounts of content across the globe with faster performance and lower latency.Mandatory JavaScript Knowledge for Coding Interview2021-12-09T00:00:00+00:002021-12-09T00:00:00+00:00https://www.rupeshtiwari.com/mandatory-javascript-knowledge-for-coding-interview<blockquote>
<p>Are you attempting coding interview challenges in Javascript? Then make sure you learn below mandatory JavaScriipt apis and utilities.</p>
</blockquote>
<h2 id="math-library">Math Library</h2>
<h3 id="mathfloor-mathround">Math.floor Math.round</h3>
<p>Used in Binary Search algorithm</p>
<p><img src="https://i.imgur.com/IZu0fyb.png" alt="" class="full" /></p>
<h3 id="mathabs">Math.abs</h3>
<p><img src="https://i.imgur.com/Fgh6Eca.png" alt="" class="full" /></p>
<h2 id="array-related">Array related</h2>
<h3 id="array-slice">Array Slice</h3>
<p>Used in Merge sort algorithm.</p>
<p><img src="https://i.imgur.com/TPf6sHD.png" alt="" class="full" /></p>
<h3 id="array-sort">Array Sort</h3>
<p><img src="https://i.imgur.com/hY8qjWD.png" alt="" class="full" /></p>
<h3 id="initialize-2d-array">Initialize 2D array</h3>
<p><img src="https://i.imgur.com/74bDGJV.png" alt="" class="full" /></p>
<h2 id="map">Map</h2>
<h3 id="javascript-map">JavaScript Map</h3>
<p><img src="https://i.imgur.com/TVb5n1i.png" alt="" class="full" /></p>
<h2 id="difference-between-i-and-i">Difference between i++ and ++i</h2>
<p>So basically ++i returns the value after it is incremented, while i++ return the value before it is incremented.</p>
<p><img src="https://i.imgur.com/mwT7aco.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/mp77XBD.png" alt="" class="full" /></p>
<h2 id="javascript-bitwise-operator">Javascript Bitwise operator</h2>
<h3 id="right-shift-xy">Right Shift x»y</h3>
<p>Moving bit/s towards the right side in binary number.</p>
<p><code class="language-plaintext highlighter-rouge">4>>2 = 16</code></p>
<p><code class="language-plaintext highlighter-rouge">x>>y</code> means <code class="language-plaintext highlighter-rouge">x/2^y</code> divide x by 2 to the power of y.</p>
<h3 id="left-shift-xy">Left Shift x«y</h3>
<p>Moving bit/s towards the left side in binary number.</p>
<p><code class="language-plaintext highlighter-rouge">4<<2 = 0</code></p>
<p><code class="language-plaintext highlighter-rouge">x<<y</code> means <code class="language-plaintext highlighter-rouge">x*2^y</code> multiply x by 2 to the power of y.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariAre you attempting coding interview challenges in Javascript? Then make sure you learn below mandatory JavaScriipt apis and utilities.What is EBIT and EBITDA in Finance?2021-12-09T00:00:00+00:002021-12-09T00:00:00+00:00https://www.rupeshtiwari.com/what-is-ebit-and-ebitda<blockquote>
<p>EBITDA and EBIT is a very fundamental financial term that is nowadays mostly considered during the cloud adoption phase for IT companies. If your company is planning to migrate to cloud and you are going to present a finance key performance index to your CFO then make sure you learn these concepts.</p>
</blockquote>
<p>Broadly for any company there exist 3 types of financial statements as following:</p>
<ol>
<li>Balance sheet</li>
<li>Income or Profit & Loss (P&L) Statement</li>
<li>Cache Flow Statement</li>
</ol>
<h2 id="what-is-a-balance-sheet">What is a Balance Sheet? </h2>
<p>Balance Sheet consists of Assets and Liabilities.</p>
<p><strong>Assets</strong> are of 2 types Fixed and Current Assets. Fixed Assets which you get the cash value after 1 year example real estate, building, equipment, vehicles. Current Assets which you convert into cash within 1 year</p>
<p><strong>Liabilities</strong> are 2 types: Long Term and Current. Long Term liabilities which you have to start paying after 1 year. Current liability which you have to pay off within 1 year.</p>
<p><img src="https://i.imgur.com/AN9wBPy.png" alt="" class="full" /></p>
<h2 id="what-is-an-income-or-profit--loss-pl-statement">What is an Income or Profit & Loss (P&L) Statement? </h2>
<p>If you subtract your company Expenses from Revenue then you get the profit.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Profit = Revenue - Expenses
</code></pre></div></div>
<p><img src="https://i.imgur.com/iMP6jRA.png" alt="" class="full" /></p>
<p>The profit that you make as cash is added to the Current Assets in your Balance Sheet. So if you make more profit your cash reserves increase that increases your assets as well.</p>
<h2 id="what-is-a-cash-flow-statement">What is a Cash Flow Statement?</h2>
<p>A Cash Flow Statement is just like your bank account statement that shows how much money is credited (deposited) or debited (withdrawn) from your bank account.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Net Cash Flows = Cash In - Cash Out
</code></pre></div></div>
<p>Cash In could be in any form of money deposit to your bank account it could be loan, income revenue, friend deposit money. Cash out could be any form of withdrawal from your account for any reason. The net balance of your cash is the net cash flows of your account.</p>
<p><img src="https://i.imgur.com/bFayiGL.png" alt="" class="full" /></p>
<p>EBIT & EBITDA is a metric of Profit and Loss statement that explains which company stock is good and which company is performing better at an operational level.</p>
<h2 id="amortization-vs-depreciation">Amortization vs Depreciation</h2>
<p>Tangible assets are physical assets that can be touched. Example of Tangible assets are as follows:</p>
<ul>
<li>Buildings</li>
<li>Equipments</li>
<li>Furnitures</li>
<li>Vehicles</li>
<li>Land</li>
<li>Machinery</li>
</ul>
<p>Depreciation is the expense of a fixed asset over its useful life.</p>
<p>Intangible assets are not physical assets, they can not be touched. Example of Intangible assets are as follows:</p>
<ul>
<li>Patents</li>
<li>Franchise agreements</li>
<li>Organizational Costs</li>
<li>Proprietary process such as copyrights like purchasing cost of software license</li>
<li>Bonds</li>
<li>Employees of a company</li>
</ul>
<p>Amortization is the practice of spreading an intangible asset’s cost over that asset’s useful life.</p>
<h2 id="what-is-non-operational-expenses">What is Non Operational Expenses?</h2>
<p>Non operational expenses are the variables that differ company by company. Majorly below 3 categories of expenses are called non-operational expenses:</p>
<ol>
<li>Interest - Depends on the company’s Financing Structure. If a company has taken a large loan then interest is high and vice-versa.</li>
<li>Taxes - Depends on Geography</li>
<li>Depreciation & Amortization - Past Investments. Non-Cash Expense</li>
</ol>
<h2 id="what-is-ebit--ebitda">What is EBIT & EBITDA? </h2>
<p>EBITDA means Earning Before Interest, Tax, Depreciation and Amortization.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>EBITDA = Gross Profit - Operating Expenses
EBIT = EBITDA - Depreciation + Amortization
PBT = EBIT - Interest
Net Profit = PBT - Tax
EBIT = Net Profit + Interest + Tax
EBITDA = Net Profit + Interest + Tax + Depreciation + Amortization
</code></pre></div></div>
<p>Vehicle Manufacturing Company EBITDA & EBIT Statement</p>
<p><img src="https://i.imgur.com/eND0t6N.png" alt="" class="full" /></p>
<h2 id="why-is-ebitda-important-for-a-company">Why is EBITDA important for a company?</h2>
<p>The Net Profit of a company doesn’t describe the financial status of a company. Because, non operational expenses such as Interest, Tax component, Depreciations, Amortizations vary per company. Example some companies might have no loan so no interest so they might show more net profit. Other companies have taken larger loans so their interest rate is high so net profit becomes low. Taxes can change as per geographical area. Depreciations may differ per company based on old/new physical assets. Therefore, you should financially compare companies at operational level only.</p>
<p>In order for creating a Level <a href="https://en.wikipedia.org/wiki/Level_playing_field">Playing Field</a> you will use EBIT & EBITDA which is used to compare companies on their operational costs. That is the recommended way to identify which company is financially stronger.</p>
<p>EBITDA is mostly considered in heavily capitalized industries like telecommunications, manufacturing, oil & gas etc. industries.</p>
<p>EBIT is mostly used for service oriented industries like Consulting and Technologies.</p>
<h2 id="why-do-c-level-executives-care-more-about-ebitda">Why do C-Level Executives care more about EBITDA?</h2>
<p>Because when you move your company to cloud then you get rid of capital expenses and incur more operational expenses. As capital expenses decrease and operational expenses increase the EBITDA value decreases. In most of the companies C-level executives like CFO, CIO, CEO get incentives, bonus based on EBITDA value. If EBITDA decreases then their bonus money gets reduced. However, IT companies also have data center staff expenses that are being calculated as the operational cost. Since that costs are now reduced or completely removed based on how many data centers your company closes. That value can balance out the EBITDA value and make your c-executives happy in terms of financial bonus money.</p>
<h2 id="conclusion">Conclusion </h2>
<p>So EBITDA is the term used by many software IT industries to publish their financial growth. You can check Microsoft Azure EBITDA margin over the last 10 years here in this chart</p>
<p><img src="https://i.imgur.com/5mJdn6U.png" alt="" class="full" /></p>
<p><a href="https://www.macrotrends.net/stocks/charts/MSFT/microsoft/ebitda-margin">microsoft ebitda margin</a></p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariEBITDA and EBIT is a very fundamental financial term that is nowadays mostly considered during the cloud adoption phase for IT companies. If your company is planning to migrate to cloud and you are going to present a finance key performance index to your CFO then make sure you learn these concepts.Introduction of Managed Identities2021-12-08T00:00:00+00:002021-12-08T00:00:00+00:00https://www.rupeshtiwari.com/introduction-of-managed-identities<blockquote>
<p>Do you have situations where a process is trying to use your virtual machine in the cloud and you want to protect and control the access? Do you want to use cosmos db using your server layer deployed over azure cloud without managing your own identity and passwords? Use Microsoft Managed Identity. In this article I will describe Managed Identities concepts.</p>
</blockquote>
<p>First let’s remind ourselves of what service principals are. Below are called Service Principal:</p>
<ol>
<li>Application principal</li>
<li>Managed Identity</li>
<li>Legacy</li>
</ol>
<h2 id="what-is-managed-identity">What is Managed Identity?</h2>
<p>Managed Identity is part of the authentication process. Managed Identity is a service principal of a special type that may only be used with Azure resources. When Managed Identity is deleted, the corresponding service principal is automatically removed. When Managed Identity is created the managed identity resource provider (MSRP) issues certificates to the identity internally. So Managed Identities are certificate based service principals.</p>
<p><img src="https://i.imgur.com/bmKv6pn.png" alt="" class="full" /></p>
<p>Managed Identities for Azure resources (formally known as MSI (Managed Service Identity) provide Azure services with an Identity in Azure Active Directory. Azure services use this identity to authenticate to services that support Azure AD authentication without needing credentials in your code.</p>
<p><img src="https://i.imgur.com/MRY8iyy.png" alt="" class="full" /></p>
<h2 id="types-of-managed-identities">Types of Managed Identities</h2>
<p>In Microsoft Azure there are 2 types of managed identities:</p>
<ol>
<li>System assigned</li>
<li>User assigned</li>
</ol>
<h2 id="system-assigned-identity">System Assigned Identity</h2>
<p><img src="https://i.imgur.com/BWz8lDS.png" alt="" class="full" /></p>
<p>Figure:- User is enabling system-assigned managed identity to the instance of an azure resource.</p>
<p>An identity is created in the azure active directory that is tied to the lifecycle of the service instance. When the service instance or resource is deleted azure deletes the corresponding identity. Only Azure resources can use this identity to request a token from Azure Active Directory. You can not share this identity with multiple resources.</p>
<p><img src="https://i.imgur.com/BHRAMhV.png" alt="" class="full" /></p>
<p>You can only enable or disable managed identities at resource level. Workloads that are contained within a single Azure Resource workload for which you need independent identities. Example: Application that works on a single Virtual Machine.</p>
<p><img src="https://i.imgur.com/aEPJ3sI.png" alt="" class="full" /></p>
<h2 id="user-assigned-managed-identities">User Assigned Managed Identities</h2>
<p>You create a managed identity as a stand-alone resource called a user-assigned managed identity. Assign it to one or more instances of an Azure service. Identity is managed separately from the resource that uses it and is not tied to the resource lifecycle. It can be shared across many azure resources. You can create, read, update, delete (CRUD) on user-assigned Managed Identity.
<img src="https://i.imgur.com/a8gGbnJ.png" alt="" class="full" /></p>
<p>For Example: Workloads that run on multiple resources and which can share a single identity. For example, workloads where resources are recycled frequently, but permissions should stay consistent like multiple virtual machines need to access the same resource. You have to assign 3 system-assigned identities in a shared azure resource.</p>
<p><img src="https://i.imgur.com/A4O695X.png" alt="" class="full" /></p>
<p>However, when you use system-assigned identity then you face 2 problemes:</p>
<ol>
<li>You have to assign multiple identities to the shared resource. Imagine if there are 100 Virtual Machines or so.</li>
<li>Whenever VM is deallocated then the corresponding managed identity will be deleted as well. So you have to assign a new system assigned managed identity of the VM when it is again provisioned. So you have to manage your own self.</li>
<li>Therefore, in this situation you should create one user-assigned managed identity that can remain always unless you delete the key. Youjust assign that key to the azure virtual machines. And the shared resource will only allow that identity to authenticate.</li>
</ol>
<p>You can perform CRUD operations on Managed Identity using below tools:</p>
<ol>
<li>ARM template</li>
<li>Azure Portal</li>
<li>Azure CLI</li>
<li>Powershell</li>
<li>REST API</li>
</ol>
<p>Which operations are allowed with Managed Identity?</p>
<p>System Assigned Managed Identity: can enable disable at resource level. Use RBAC to grant permissions. CRUD on Azure Activity Logs view Sign-In activity in Azure AD sign-in logs.</p>
<p>UserAssigned Managed Identity: you can do CRUD on identities. Use RBAC to grant permissions. You can use one or more than 1 resources.CRUD on Azure Activity log view sign-in activity in Azure AD sign-in logs.</p>
<h2 id="references">References </h2>
<ol>
<li><a href="https://docs.microsoft.com/en-us/azure/active-directory/authentication/overview-authentication">https://docs.microsoft.com/en-us/azure/active-directory/authentication/overview-authentication</a></li>
<li><a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals">https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals</a></li>
</ol>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariDo you have situations where a process is trying to use your virtual machine in the cloud and you want to protect and control the access? Do you want to use cosmos db using your server layer deployed over azure cloud without managing your own identity and passwords? Use Microsoft Managed Identity. In this article I will describe Managed Identities concepts.What is Azure Virtual Machine Scale Sets2021-12-08T00:00:00+00:002021-12-08T00:00:00+00:00https://www.rupeshtiwari.com/what-is-azure-virtual-machine-scale-sets<blockquote>
<p>Now a days distributed architecture is common. We deploy our services into many different servers to scale them up and meet our demand. However, managing all servers for <strong>load-balancing, scaling, make application highly available</strong> is super challenging on cloud. <strong>Azure Virtual Machine Scale sets</strong> is the great tool which does all of these automatically with <strong>no extra cost</strong> for you. Lets learn more about Scale sets in this article.</p>
</blockquote>
<p>For Black Friday sell you want your services to be running in 50 instances and on regular day you want 5 instances. So in cloud Virtual machine scale set you get automatic scale up and down of your VM instances.</p>
<h2 id="scalability-vs-elasticity">Scalability vs Elasticity</h2>
<p>Scalability means to increase from 5 to 50 instances. Elasticity is to reduce 50 instances to 5. With VMSS scalability and elasticity is possible automatically.</p>
<h2 id="vertical-vs-horizontal-scaling">Vertical vs Horizontal Scaling</h2>
<p><img src="https://imgur.com/EkHkAX3.png" alt="" class="full" /></p>
<p>Two types of scaling vertical and horizontal.</p>
<h3 id="vertical-scaling-or-scale-updown">Vertical Scaling or Scale Up/Down</h3>
<p>In Vertical scaling if you have 1 core CPU and 4GB memory and if you can upgrade this configuration to 2 core CPU and 8 GB memory. This is called as Scale UP. You can reduce the configuration back to baseline which is called as Scale Down.</p>
<h3 id="horizontal-scaling-or-scale-outin">Horizontal Scaling or Scale Out/In</h3>
<p>Horizontal Scaling is you start with one VM and you keep adding more VM with same configuration is also called as Scale Out. You can also reduce back to 1 VM once your Sale or Business demand is over this process is called as Scale In.</p>
<h2 id="distributed-architecture-with-scale-sets">Distributed Architecture with Scale Sets</h2>
<p>You must need <strong>Distributed Architecture</strong> to provide <strong>redundancy and improved performance</strong>, applications are typically <strong>distributed across multiple instances</strong>. In order to give your customer faster speed and low latency you may need <strong>load balancer</strong> that <strong>distributes requests</strong> to one of the application <strong>instances</strong>.</p>
<h2 id="what-is-azure-virtual-machine-scale-sets">What is Azure Virtual Machine Scale Sets?</h2>
<p><img src="https://imgur.com/3qqKpF8.png" alt="" class="full" /></p>
<p>Virtual Machine Scale Set (VMSS) let you <strong>Create</strong> and <strong>Manage</strong> a <strong>group of load balanced VMs</strong>. The number of VM instances can <strong>automatically increase or decrease</strong> in response to demand or a defined schedule.</p>
<ul>
<li>With virtual machine scale sets, you can build <strong>large-scale services for areas such as compute, big data, and container workloads</strong>.</li>
<li>Scale sets provide <strong>high availability to your applications</strong>, and allow you to <strong>centrally manage</strong>, <strong>configure</strong>, and <strong>update</strong> a large number of VMs.</li>
<li><strong>Consistent Configuration</strong>: Virtual Machine scale sets are the objects that are used to run multiple instances of your application and maintain a consistent configuration across your environment.</li>
</ul>
<p class="notice--info">VMs in a scale set are identical, so you can create them from the same base operating system image.</p>
<h2 id="how-virtual-machine-scale-set-vmss-works">How Virtual Machine Scale Set (VMSS) works</h2>
<p>VMSS uses <a href="#minimum-instance">minimum instance</a> to start with and you can set the <a href="#maximum-instance">maximum instance</a> of your virtual machine. You can setup rules based on <a href="#time-based-scaling">Time</a>, <a href="#metrics">Metrics</a> based to increase or decrease VM instances. VMSS has in build <a href="#load-balancer-in-scale-set">load balancers</a>. Public load balancer works with internet traffic to your VMs. Which looks upon the CPU metrics and if CPU utilization is more than 75% then wait for some time and add another VM instance without any manual steps required.</p>
<h3 id="minimum-instance">Minimum Instance</h3>
<p>This defines how much instance of VM minimum you need. You can set it to 1.</p>
<h3 id="maximum-instance">Maximum Instance</h3>
<p>You can go up to 1000 VM instances. For example for your business you can start with 1 VM and scale out up to 3 instances. Therefore, you set Maximum instance value to 3. Therefore, even though the demand is high Azure will not spin up 4th instance. It will stay up to 3 instance only and you save budget.</p>
<h3 id="metrics-based-scaling">Metrics Based Scaling</h3>
<p>If you don’t know where you are going to get the maximum business, it may be today, next day or any 5 continuous days then you must go for Metrics based scaling. You can set rule like If my VM CPU utilization is > 75% then add one more VM if it is less than ( < ) 25% then remove the VM.</p>
<p><strong>Manually increasing instances</strong>: If you know tomorrow only you want to increase your VM instance to 4 instances. So you do that manually and revert it on the day after tomorrow.</p>
<h3 id="time-based-scaling">Time Based Scaling</h3>
<p><strong>Custom</strong> : You can do <strong>Time Based</strong> increment or decrement Here you can schedule your VMs to scale out and scale in. For example every Saturday increase the VM instance to 4 and on Sunday reduce it back to 1. You can schedule these rules.</p>
<h3 id="load-balancer-in-scale-set">Load Balancer in Scale Set</h3>
<p><img src="https://imgur.com/8H6dwMy.png" alt="" class="full" /></p>
<p>An <strong>Azure load balancer</strong> is a Layer-4 (TCP, UDP) <strong>load balancer</strong> that provides high availability by distributing incoming traffic among healthy VMs. A <strong><a href="https://docs.microsoft.com/en-us/azure/load-balancer/components#frontend-ip-configurations">public load balancer</a></strong> can provide outbound connections for virtual machines (VMs) inside your virtual network. Public Load Balancers are used to load balance internet traffic to your VMs. An <strong><a href="https://docs.microsoft.com/en-us/azure/load-balancer/components#frontend-ip-configurations">internal (or private) load balancer</a></strong> is used where private IPs are needed at the frontend only. Internal load balancers are used to load balance traffic inside a virtual network.</p>
<h2 id="virtual-machine-scale-set-architecture">Virtual Machine Scale Set Architecture</h2>
<p>Virtual Machine Scale Set (VMSS) deploys Virtual Machines in a single subnet of a Virtual Network. Figure below shows Architecture of Virtual Machine Scale Set (VMSS) deployed in single subnet with Single Placement group.</p>
<p><img src="https://imgur.com/XUk0AFB.png" alt="" class="full" /></p>
<p><strong>Virtual Network</strong> with Single Subnet is automatically created during VMSS deployment. All virtual machines specified in create VMSS blade are deployed in the Single Subnet.</p>
<p><strong>Azure Load Balancer</strong> is automatically created during VMSS deployment.</p>
<p><strong>Placement group</strong> is availability set with five fault domains and five update domains and support up to 100 VMs. Placement group is automatically created by VMSS. Additional Placement groups will be automatically created by VMSS if you are deploying more than 100 instances.</p>
<p><strong>Storage</strong>: VMSS can use managed disks or unmanaged disks for Virtual Machine storage. Managed disks are required to create more than 100 Virtual Machines. Unmanaged disks are limited to 100 VMs and single Placement Group.</p>
<p><strong>VM Diagnostic Logs</strong> are guest OS performance counters and are streamed to Azure storage Account. VMSS uses this data for making Auto-scaling decisions.</p>
<h2 id="why-use-virtual-machine-scale-sets">Why use Virtual Machine Scale Sets?</h2>
<ul>
<li><strong>Maintenance Mode Support</strong>: If you need to perform maintenance or update an application instance, your customers must be distributed to another available application instance.</li>
<li><strong>Automatically increase VM instances</strong>: To keep up with additional customer demand, you may need to increase the number of application instances that run your application.
<ul>
<li>Auto scale based on metrics.</li>
<li>Auto scale based on a defined schedule. Suppose starting next week you are going to have a heavy peak, for next 3 days. You can define a set schedule. For example, at 9 am on Jan 2nd 2021, increase the VM instance count to 50. And at 9PM on Jan 5th when your peak ends, bring the instance count back to your baseline configuration.</li>
</ul>
</li>
<li><strong>Easy to create and manage multiple VMs</strong> : It maintains a consistent configuration (VM size, disk configuration) across your environment. All VM instances are created from the same base OS image and configuration. This approach lets you easily manage hundreds of VMs without additional configuration tasks or network management. For basic layer-4 traffic distribution it uses Azure Load Balancer. And for advanced layer-7 it uses Azure Application Gateway.</li>
<li><strong>Provides high availability and application resiliency</strong>: If one of these VM instances has a problem, customers continue to access your application through one of the other VM instances with minimal interruption.</li>
<li><strong>Allows your application to automatically scale as resource demand changes</strong>: Like it auto increase VM instances. It can also minimizes the number of unnecessary VM instances that run your application when demand is low, while customers continue to receive an acceptable level of performance</li>
<li><strong>Works at large-scale</strong> : Up to <strong>1000</strong> Azure VM, and custom VM images up to <strong>600</strong> VM.</li>
</ul>
<h3 id="scale-set-saves-money-">Scale set saves money 💰</h3>
<p>The <strong>management</strong> and <strong>automation</strong> features, such as auto-scale and redundancy, incur <strong>no additional charges</strong> over the use of VMs. You only pay for the underlying compute resources such as the VM instances, load balancer, or Managed Disk storage.</p>
<h3 id="benefits-of-scale-set">Benefits of Scale Set</h3>
<table>
<thead>
<tr>
<th style="text-align: left">scenario</th>
<th style="text-align: left">Manage VMs manually</th>
<th style="text-align: right">Use VM Scale Set</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left">Adding extra VM</td>
<td style="text-align: left">Manually create, configure and ensure compliance</td>
<td style="text-align: right">Automatically create from central configuration</td>
</tr>
<tr>
<td style="text-align: left">Traffic Balancing & distribution</td>
<td style="text-align: left">Manually</td>
<td style="text-align: right">Automatically create and integrate Azure load balancer or Application Gateway</td>
</tr>
<tr>
<td style="text-align: left">High availability and redundancy</td>
<td style="text-align: left">Manually create Availability set or distribute and track VMs across Availability Zones</td>
<td style="text-align: right">Automatic distribution of VM instances across Availability Zones or Availability Sets</td>
</tr>
<tr>
<td style="text-align: left">Scaling of VMs</td>
<td style="text-align: left">Manual monitoring and Azure Automation</td>
<td style="text-align: right">Auto scale based on host metrics, in-guest metrics, Application Insights, or schedule</td>
</tr>
</tbody>
</table>
<p>Using scale set is a wiser decision with zero additional cost!</p>
<h2 id="building-azure-vm-using-azure-vm-image-builder">Building Azure VM using Azure VM Image Builder</h2>
<p><img src="https://imgur.com/Ut26mIt.png" alt="" class="full" /></p>
<p>Using <a href="https://docs.microsoft.com/azure/virtual-machines/linux/image-builder-overview">AZURE VM IMAGE BUILDER SERVICE</a>, you can quickly start building standardized images without needing to set up your own imaging pipeline. Just provide a simple configuration describing your image, submit it to the Image Builder service, and the image is built and distributed. You will incur some compute, networking and storage costs when creating, building and storing images with Azure Image Builder. These costs are similar to the costs incurred in manually creating custom images.</p>
<h2 id="distribute-your-vm-images">Distribute your VM images</h2>
<p><img src="https://imgur.com/2diOKki.png" alt="" class="full" /></p>
<p><a href="https://docs.microsoft.com/en-us/azure/virtual-machines/shared-image-galleries">SHARED IMAGE GALLERY</a> enables image distribution to different users, service principals, or AD groups, across multiple subscriptions within your organization and regions through a centralized image management platform.</p>
<p>Run below command for creating shared image gallery:</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">`</span><span class="n">New-AzGallery</span><span class="w"> </span><span class="nt">-GalleryName</span><span class="w"> </span><span class="err">“</span><span class="nx">azdemoGallery</span><span class="err">”</span><span class="w"> </span><span class="nt">-ResourceGroupName</span><span class="w"> </span><span class="err">“</span><span class="nx">Azure-demo</span><span class="err">”</span><span class="w"> </span><span class="nt">-Location</span><span class="w"> </span><span class="err">“</span><span class="nx">West</span><span class="w"> </span><span class="nx">US</span><span class="err">”</span><span class="se">`
</span></code></pre></div></div>
<p>Next, we need to create Gallery definition for storing windows images:</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">New-AzGalleryImageDefinition</span><span class="w"> </span><span class="nt">-GalleryName</span><span class="w"> </span><span class="err">“</span><span class="nx">azdemogallery</span><span class="err">”</span><span class="w"> </span><span class="nt">-ResourceGroupName</span><span class="w"> </span><span class="err">“</span><span class="nx">Azure-demo</span><span class="err">”</span><span class="w"> </span><span class="nt">-location</span><span class="w"> </span><span class="err">“</span><span class="nx">West</span><span class="w"> </span><span class="nx">US</span><span class="err">”</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="err">“</span><span class="nx">winserverimages</span><span class="err">”</span><span class="w"> </span><span class="nt">-OsState</span><span class="w"> </span><span class="nx">generalized</span><span class="w"> </span><span class="nt">-Ostype</span><span class="w"> </span><span class="nx">windows</span><span class="w"> </span><span class="nt">-Publisher</span><span class="w"> </span><span class="nx">demo</span><span class="w"> </span><span class="nt">-offer</span><span class="w"> </span><span class="nx">windows</span><span class="w"> </span><span class="nt">-sku</span><span class="w"> </span><span class="err">‘</span><span class="nx">win2016</span><span class="err">’</span><span class="w">
</span></code></pre></div></div>
<h2 id="how-to-monitor-scale-sets">How to monitor scale sets</h2>
<ul>
<li>Use <a href="https://docs.microsoft.com/en-us/azure/azure-monitor/vm/vminsights-overview">Azure Monitor for VMs</a>, to see metrics about VM CPU, memory, disk and network performance</li>
<li>Enable monitoring <a href="https://docs.microsoft.com/en-us/azure/azure-monitor/app/azure-vm-vmss-apps">virtual machine scale set application</a> with Application Insights to collect detailed information about your application including page views, application requests, and exceptions.</li>
<li>Configure <a href="https://docs.microsoft.com/en-us/azure/azure-monitor/app/monitor-web-app-availability">availability test</a> to check application availability to simulate user traffic.</li>
</ul>
<p class="notice--success">Learn more about <a href="https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/overview?context=/azure/virtual-machines/context/context">Azure Virtual Machine Scale Set</a></p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariNow a days distributed architecture is common. We deploy our services into many different servers to scale them up and meet our demand. However, managing all servers for load-balancing, scaling, make application highly available is super challenging on cloud. Azure Virtual Machine Scale sets is the great tool which does all of these automatically with no extra cost for you. Lets learn more about Scale sets in this article.Azure Active Directory Authentication Basics2021-12-07T00:00:00+00:002021-12-07T00:00:00+00:00https://www.rupeshtiwari.com/azure-active-directory-authentication<blockquote>
<p>Azure Active Directory (AD) verifies the credentials that are username and password. It belongs to the Azure Identity Platform.</p>
</blockquote>
<h2 id="what-is-authentication">What is Authentication?</h2>
<p>Authentication (AuthN) is the process to prove that you are who you say you are. Microsoft identity platform uses <a href="https://openid.net/connect">open Id connect protocol</a> for handling authentication.</p>
<h2 id="what-is-authorization">What is Authorization?</h2>
<p>Authorization (AuthZ) is the act of granting an authenticated party permission to do something. Microsoft identity platform uses <a href="https://oauth.net/2/">OAuth2.0</a> protocol for handling authorization.</p>
<h2 id="what-are-security-tokens">What are security Tokens?</h2>
<p>Microsoft identity provider authenticates users and provides security toneks as <a href="https://jwt.io/">JWT</a> that contains claims .</p>
<p>There are 3 types of <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/access-tokens">Security tokens:</a></p>
<ol>
<li>ID token</li>
<li>Access tokens</li>
<li>Refresh tokens</li>
</ol>
<p><img src="https://i.imgur.com/2BLbcE1.png" alt="" class="full" /></p>
<p><strong>Id token</strong>: is issued to the client during open id connect flow. ID token is used to authenticate users. It is provided by the authorized endpoint of the authorization server. It contains claims pertaining to the authentication of the end-user resource owner.</p>
<p><strong>Access token</strong>: is issued by authorization server as a part of OAuth2.0 flow. It has information about the user & resource for which the token is issued. Access token enables clients to securely call protected web APIs and are used by web api to do authentication & authorization.</p>
<p><strong>Refresh token</strong>: is issued by authorization server at the same time access token is issued. The access token expires in a short time. Client uses refresh token to get next access token.</p>
<h2 id="validating-security-token">Validating Security Token</h2>
<p>Authorization server signs the security token with a private key; it also publishes a public key. Web api verifies the signature of the token using public key. Client passess access token to the web api as bearer token in the authorization header.</p>
<p><img src="https://i.imgur.com/PCRgN6Y.png" alt="" class="full" /></p>
<h2 id="security-principal">Security Principal </h2>
<p>Security Principal is a fancy name for various types of security tokens. Below are all called as security principal:</p>
<ol>
<li>User Principal</li>
<li>Group Principal</li>
<li>Service Principal</li>
<li>Managed Identities</li>
</ol>
<h2 id="what-is-claim">What is Claim?</h2>
<p>A claim provides assertions about one entity (client application) to another entity (Resource Server). Claim contains facts about the security principal that was authenticated by the authorization server. Claim provides info about below things:</p>
<ol>
<li>Security Token Server ( who generated )</li>
<li>Date of token creation</li>
<li>Subject</li>
<li>Audience - Target application for which token generated</li>
</ol>
<h2 id="what-is-an-application-model">What is an Application Model?</h2>
<p>Azure AD has identity service and for identity provider to know which user has access to application you must register both user and application in the identity provider. This is the application model.</p>
<h2 id="microsoft-authentication-protocols">Microsoft Authentication Protocols</h2>
<p>The Microsoft Identity platform uses the OPENID connect protocol for handling authentication and OAuth2.0 protocol for authorization.
<img src="https://i.imgur.com/jQBYg3z.png" alt="" class="full" /></p>
<p>Application Registration gives application Id and redirect URI.</p>
<h2 id="what-is-bearer-token">What is Bearer Token?</h2>
<p>Bearer token is a lightweight security token that gives bearer access to protected resources.</p>
<p><img src="https://i.imgur.com/AbJl7rL.png" alt="" class="full" /></p>
<h2 id="security-tokens">Security Tokens</h2>
<p>There are 3 types of security tokens:</p>
<ol>
<li>Access tokens</li>
<li>ID Token</li>
<li>Refresh Tokens</li>
</ol>
<p><img src="https://i.imgur.com/0Da1wDj.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/81Mps2M.png" alt="" class="full" />
<img src="https://i.imgur.com/LDRXLUo.png" alt="" class="full" /></p>
<h2 id="what-is-openid-connect-protocol">What is OpenId Connect Protocol? </h2>
<p>OpenId Connect authentication protocol built on OAuth. You add authentication to your web app so that it can sign in users. Adding authentication enables your web app to access limited profile information in order to customize the experience for users.</p>
<p>Web apps authenticate a user in a web browser. In this scenario, the web app directs the user’s browser to sign them in to Azure Active Directory (Azure AD). Azure AD returns a sign-in response through the user’s browser, which contains claims about the user in a security token.
<img src="https://i.imgur.com/RbLR6P2.png" alt="" class="full" /></p>
<h3 id="retrieve-access-token">Retrieve Access Token </h3>
<p><img src="https://i.imgur.com/cquFCVf.png" alt="" class="full" /></p>
<p>First, the application needs to obtain an authorization code from Azure AD /authorize endpoint. The authorization code can then be used to acquire a new access and refresh token. If the user is not already authenticated, Azure AD /authorize endpoint prompts the user to sign in.</p>
<p><img src="https://i.imgur.com/7zQ9Q1Y.png" alt="" class="full" /></p>
<p>whether the user (or their organization’s administrator) has already granted the application consent. If consent has not already been granted, Azure AD prompts the user for consent</p>
<p><img src="https://i.imgur.com/jMNYK8q.png" alt="" class="full" /></p>
<p>After the user grants consent, an authorization code is returned to your application, which is redeemed to acquire an access token and refresh token.</p>
<p>As an administrator, you can also consent to an application’s delegated permissions on behalf of all the users in your tenant.</p>
<p><img src="https://i.imgur.com/dRWMdL4.png" alt="" class="full" /></p>
<h2 id="azure-active-directory-authentication">Azure Active Directory Authentication</h2>
<p>Azure Active Directory has below components for authentication:</p>
<ol>
<li>Self-Service Password Reset</li>
<li>Azure AD Multi-Factor authentication</li>
<li>Hybrid Integration to write password changes back to on-premises</li>
<li>Hybrid integration to enforce password protection policy for an on-premise environment</li>
<li>Passwordless authentication</li>
</ol>
<h2 id="azure-ad-self-service-password-reset">Azure AD Self-Service Password Reset</h2>
<p>You can change or reset passwords without any admin help. Below are the self-services provided:</p>
<ol>
<li>Password changes</li>
<li>Password reset</li>
<li>Account unlock</li>
</ol>
<p>All of the above activities can be done on cloud Azure AD & these are written back to the on-premises AD. So if you change password it will be synced to on-premises AD.</p>
<h2 id="azure-ad-multi-factor-authentication">Azure AD Multi-Factor Authentication</h2>
<p>During authentication it will prompt additional information.
<img src="https://i.imgur.com/L6gNomV.png" alt="" class="full" /></p>
<p>Multi-factor authentication needs 2 or more information from below list:</p>
<ol>
<li>Something you know :- password</li>
<li>Something you have :- Mobile or hardware key</li>
<li>Something you are :- biometric (fingerprint, face scan )</li>
</ol>
<p><img src="https://i.imgur.com/1W7asfs.png" alt="" class="full" /></p>
<h2 id="password-protection-by-azure-ad">Password Protection by Azure AD</h2>
<p>To enforce the use of strong passwords. Azure AD blocks weak passwords like (password123) etc. You can define a custom password policy for your organization. With hybrid integration you can even synchronize policies and weak/banned password list on-premise & enforce them.</p>
<p><img src="https://i.imgur.com/oN6DpHN.png" alt="" class="full" /></p>
<h2 id="passwordless-authentication-by-azure-ad">Passwordless Authentication by Azure AD</h2>
<p>You provide credentials using</p>
<ol>
<li>Biometric with windows hello for business</li>
<li>FIDO2 security key</li>
</ol>
<p>These authentication methods can not be easily duplicated by attackers therefore, they are highly safe.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full full" /></p>Rupesh TiwariAzure Active Directory (AD) verifies the credentials that are username and password. It belongs to the Azure Identity Platform.Create Update and Delete Tags on Azure Resource2021-12-06T00:00:00+00:002021-12-06T00:00:00+00:00https://www.rupeshtiwari.com/create-update-and-delete-tags-on-azure-resource<blockquote>
<p>Learn more about how to use tags to organize your Azure resources and management hierarchy. Also learn the <code class="language-plaintext highlighter-rouge">cmdlet</code> used to fetch tags from Azure resources.</p>
</blockquote>
<h2 id="what-is-tag-in-azure-resources">What is Tag in Azure Resources?</h2>
<ul>
<li>You apply tags to your <strong>Azure resources, resource groups, and subscriptions</strong> to <code class="language-plaintext highlighter-rouge">logically</code> organize them into a <code class="language-plaintext highlighter-rouge">taxonomy</code>.</li>
<li>Each tag consists of a <strong>name and a value pair</strong>. For example, you can create tag and apply the name “Environment” and the value “Stage” to all the resources in stage. Learn more about <a href="https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/tag-resources">Tagging ideas</a></li>
<li><strong>Tag names are case-insensitive</strong>: Example: “Environment” or “environment” they are same tag names.</li>
<li><strong>Tag values are case-sensitive</strong>: Example: “Stage” or “stage” they are different tag values.</li>
</ul>
<p class="notice-success">Learn more about <a href="https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/tag-resources">how to use tags to organize your Azure resources and management hierarchy</a></p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Get-AzResource</span><span class="w"> </span><span class="c"># To fetch all resources</span><span class="w">
</span></code></pre></div></div>
<h2 id="apply-tags-to-azure-resource">Apply tags to Azure Resource</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/QTDDL8_0N3c" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p>Use <code class="language-plaintext highlighter-rouge">New-AzTag</code> to apply tag on resource in <strong>Azure</strong>.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Create Tag variable</span><span class="w">
</span><span class="nv">$tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="s2">"Dept"</span><span class="o">=</span><span class="s2">"Finance"</span><span class="p">;</span><span class="w"> </span><span class="s2">"Status"</span><span class="o">=</span><span class="s2">"Normal"</span><span class="p">}</span><span class="w">
</span><span class="c"># Get the resource that you want to appply tag on.</span><span class="w">
</span><span class="nv">$resource</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-AzResource</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s2">"Application Insights Smart Detection"</span><span class="w"> </span><span class="nt">-ResourceGroup</span><span class="w"> </span><span class="nx">AdmissionApplication</span><span class="w">
</span><span class="c"># Apply Tag</span><span class="w">
</span><span class="n">New-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w"> </span><span class="nt">-Tag</span><span class="w"> </span><span class="nv">$tags</span><span class="w">
</span></code></pre></div></div>
<h2 id="view-tag-on-azure-resource">View Tag on Azure Resource</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/QTDDL8_0N3c" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p>Use <code class="language-plaintext highlighter-rouge">Get-AzTag</code> cmdlet to view tag on Azure resource. You have to pass the resource Id though. You can also use <code class="language-plaintext highlighter-rouge">Get-AzResource</code> cmdlet to view the tags directly under a particular resource.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Get the resource that you want to view tag on.</span><span class="w">
</span><span class="nv">$resource</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-AzResource</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s2">"Application Insights Smart Detection"</span><span class="w"> </span><span class="nt">-ResourceGroup</span><span class="w"> </span><span class="nx">AdmissionApplicationGet-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w">
</span><span class="c"># View tags</span><span class="w">
</span><span class="n">Get-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w">
</span></code></pre></div></div>
<h2 id="delete-tag-on-a-azure-resource">Delete Tag on a Azure Resource</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/2211KACFlmw" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p>To remove specific tags, use <strong>Update-AzTag</strong> and set <strong>-Operation</strong> to <strong>Delete</strong>. Pass in the tags you want to delete.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Get the resource that you want to view tag on.</span><span class="w">
</span><span class="nv">$resource</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-AzResource</span><span class="w"> </span><span class="nt">-Name</span><span class="w"> </span><span class="s2">"Application Insights Smart Detection"</span><span class="w"> </span><span class="nt">-ResourceGroup</span><span class="w"> </span><span class="nx">AdmissionApplication</span><span class="w">
</span><span class="c"># View tags</span><span class="w">
</span><span class="n">Get-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w">
</span><span class="c"># Decide which tags to remove</span><span class="w">
</span><span class="nv">$removeTags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="s2">"Dept"</span><span class="o">=</span><span class="s2">"Finance"</span><span class="p">;</span><span class="w"> </span><span class="s2">"Status"</span><span class="o">=</span><span class="s2">"Normal"</span><span class="p">}</span><span class="w">
</span><span class="c"># Delete the tags</span><span class="w">
</span><span class="n">Update-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w"> </span><span class="nt">-Tag</span><span class="w"> </span><span class="nv">$removeTags</span><span class="w"> </span><span class="nt">-Operation</span><span class="w"> </span><span class="nx">Delete</span><span class="w">
</span><span class="c"># Confirm Tags are deleted by fetching them again</span><span class="w">
</span><span class="n">Get-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w">
</span></code></pre></div></div>
<h2 id="delete-tag-on-a-azure-subscription">Delete Tag on a Azure Subscription</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/2211KACFlmw" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p>To remove all tags from your Azure subscription, use the <a href="https://docs.microsoft.com/en-us/powershell/module/az.resources/remove-aztag">Remove-AzTag</a> command.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Get subscription by it's name</span><span class="w">
</span><span class="nv">$subscription</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">Get-AzSubscription</span><span class="w"> </span><span class="nt">-SubscriptionName</span><span class="w"> </span><span class="s2">"Example Subscription"</span><span class="p">)</span><span class="o">.</span><span class="nf">Id</span><span class="w">
</span><span class="c"># Remove the tags</span><span class="w">
</span><span class="n">Remove-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="s2">"/subscriptions/</span><span class="nv">$subscription</span><span class="s2">"</span><span class="w">
</span></code></pre></div></div>
<h2 id="merge-tag">Merge Tag</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/8vtnn1ZacNg" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="s2">"Location"</span><span class="o">=</span><span class="s2">"Canada"</span><span class="p">}</span><span class="w">
</span><span class="c"># Merge Tag</span><span class="w">
</span><span class="n">Update-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w"> </span><span class="nt">-Tag</span><span class="w"> </span><span class="nv">$tags</span><span class="w"> </span><span class="nt">-Operation</span><span class="w"> </span><span class="nx">Merge</span><span class="w">
</span></code></pre></div></div>
<h2 id="replace-tag">Replace Tag</h2>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/0GJ27K3CWXA" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">@{</span><span class="s2">"Dept"</span><span class="o">=</span><span class="s2">"Sales"</span><span class="p">;</span><span class="w"> </span><span class="s2">"Status"</span><span class="o">=</span><span class="s2">"High"</span><span class="p">}</span><span class="w">
</span><span class="c"># Replace tags</span><span class="w">
</span><span class="n">Update-AzTag</span><span class="w"> </span><span class="nt">-ResourceId</span><span class="w"> </span><span class="nv">$resource</span><span class="o">.</span><span class="nf">id</span><span class="w"> </span><span class="nt">-Tag</span><span class="w"> </span><span class="nv">$tags</span><span class="w"> </span><span class="nt">-Operation</span><span class="w"> </span><span class="nx">Replace</span><span class="w">
</span></code></pre></div></div>
<p class="notice--info"><i class="fab fa-youtube" color="primary"></i> <br />
Watch the entire videos on <a href="https://www.youtube.com/playlist?list=PLZed_adPqIJrUamBBcr9rYC6GFqbWK1mG"><strong>Full Stack Master YouTube Channel</strong> related to Managing Tags in Azure Resources</a></p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariLearn more about how to use tags to organize your Azure resources and management hierarchy. Also learn the cmdlet used to fetch tags from Azure resources.Introduction to MSAL2021-12-05T00:00:00+00:002021-12-05T00:00:00+00:00https://www.rupeshtiwari.com/what-is-msal<blockquote>
<p>If you have any project that you want to integrate with Azure Active Directory and secure your app. Then you must try using Microsoft Authentication Library (MSAL). Yes, in this article I will introduce you to MSAL 🔥.</p>
</blockquote>
<h2 id="what-is-msal">What is MSAL?</h2>
<p>Microsoft Authentication Library (MSAL) gives secure access to acquire security token on-behalf of user or application and helps you to achieve single sign on.
<img src="https://i.imgur.com/zb53M5t.png" alt="" /></p>
<p>MSAL specifies which audience can sign in to your application.</p>
<h2 id="what-is-the-audience">What is the audience?</h2>
<p>Audience is the actor or client who interact with the system to whom your application authenticate. Sometimes audience can be used for the target application for which security principal is generated. Audience could be:</p>
<ol>
<li>Your organization</li>
<li>Several Organization</li>
<li>Work Organization</li>
<li>School Organization</li>
<li>Microsoft Personal account</li>
<li>Social Identities with Azure AD B2B</li>
<li>Users in Sovereign</li>
<li>National Cloud (Sovereign cloud )
<ol>
<li>Azure Government</li>
<li>Azure Germany</li>
<li>Azure China 21Vianet</li>
</ol>
</li>
</ol>
<h2 id="which-applications-msal-supports">Which Applications MSAL supports?</h2>
<p>In your application you do not need to write code, you just install the MSAL library and configure it. MSAL will get the security tokens from Azure Identity provider for</p>
<ol>
<li>Web applications</li>
<li>Web APIs</li>
<li>SPA (Javascript)</li>
<li>Mobile & Native Applications</li>
<li>Daemons & Server Side applications</li>
</ol>
<h2 id="difference-between-adal--msal">Difference between ADAL & MSAL </h2>
<p>Active Directory Authentication Library (ADAL) integrates with the Azure AD for developers (v1.0) endpoint, where MSAL integrates with the Microsoft identity platform.</p>
<p><img src="https://i.imgur.com/mF36jCV.png" alt="" /></p>
<h2 id="what-programming-language-msal-supports">What Programming Language MSAL supports?</h2>
<p>MSAL can be used with below languages:</p>
<ol>
<li>.net applications</li>
<li>Angular projects</li>
<li>JAVA applications</li>
<li>Javascript apps</li>
<li>Node.js projects</li>
<li>Python</li>
<li>React</li>
<li>Android</li>
</ol>
<p>This list is continuously growing please check with Microsoft document for more updates.</p>
<p>MSAL used for asp.net web applications is called as Microsoft Identity Web Library</p>
<h2 id="references">References</h2>
<ol>
<li><a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-overview#differences-between-adal-and-msal">https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-overview#differences-between-adal-and-msal</a></li>
</ol>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariIf you have any project that you want to integrate with Azure Active Directory and secure your app. Then you must try using Microsoft Authentication Library (MSAL). Yes, in this article I will introduce you to MSAL 🔥.Site Recovery Strategy in Azure2021-12-04T00:00:00+00:002021-12-04T00:00:00+00:00https://www.rupeshtiwari.com/site-recoverystrategy-in-azure<blockquote>
<p>Learn the failover and failback stages in Site Recovery on Azure. Learn <strong>Business Continuity and Disaster Recovery</strong> (BCDR) strategy in Azure.</p>
</blockquote>
<h2 id="failover-and-failback-in-site-recovery-has-four-stages">Failover and failback in Site Recovery has four stages</h2>
<p><img src="https://imgur.com/TO8eYHW.png" alt="" /></p>
<p><strong>Stage 1: Fail over from on-premises:</strong> After setting up replication to Azure for on-premises machines, when your on-premises site goes down, you fail those machines over to Azure. After failover, Azure VMs are created from replicated data.</p>
<p><strong>Stage 2: Reprotect Azure VMs:</strong> In Azure, you reprotect the Azure VMs so that they start replicating back to the on-premises site. The on-premises VM (if available) is turned off during reprotection, to help ensure data consistency.</p>
<p><strong>Stage 3: Fail over from Azure:</strong> When your on-premises site is running as normal again, you run another failover, this time to fail back Azure VMs to your on-premises site. You can fail back to the original location from which you failed over, or to an alternate location.</p>
<p><strong>Stage 4: Reprotect on-premises machines:</strong> After failing back, again enable replication of the on-premises machines to Azure.</p>
<p><img src="https://imgur.com/Zuy3Ijz.png" alt="" /></p>
<p class="notice--success">Learn more about <a href="https://docs.microsoft.com/en-us/azure/site-recovery/failover-failback-overview#recovery-stages">BCDR strategy in Azure here</a>.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is waiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariLearn the failover and failback stages in Site Recovery on Azure. Learn Business Continuity and Disaster Recovery (BCDR) strategy in Azure.On-Premises Disaster Site Recovery Strategy with Azure VM2021-12-03T00:00:00+00:002021-12-03T00:00:00+00:00https://www.rupeshtiwari.com/on-premises-disaster-site-recovery-strategy-in-azure<blockquote>
<p>Site Recovery is very critical now a days. In your own Data-center or Server disasters can happen and you must secure your site to recover from them. Use Microsoft Azure to secure you on-premises machine today. Learn the failover and failback stages in Site Recovery on Azure. Learn <strong>Business Continuity and Disaster Recovery</strong> (BCDR) strategy in Azure.</p>
</blockquote>
<h2 id="setup-on-premises-replication-to-azure-vm">Setup On-Premises Replication to Azure VM</h2>
<p>Before even failover to Azure VM from on-premises machine make sure you have setup your replication to Azure VM.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/zU1glIxnPnM" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p><img src="https://imgur.com/CEHLtIe.png" alt="" class="full" /></p>
<h2 id="stages-for-on-premises-disaster-recovery">Stages for On-Premises Disaster Recovery</h2>
<p><img src="https://imgur.com/TO8eYHW.png" alt="" class="full" /></p>
<p>After replication is setup to Azure VM follow this workflow to accomplish Site Recovery in 4 steps.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/rb7mRDol2hs" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<h2 id="on-premises-failover-and-failback-four-stages">On-Premises Failover and failback four stages</h2>
<p>Let’s learn all the 4 steps required for site recovery.</p>
<h3 id="stage-1-fail-over-from-on-premises">Stage 1: Fail over from on-premises</h3>
<p><img src="https://imgur.com/AdzMjJk.png" alt="" class="full" /></p>
<p>After setting up replication to Azure for on-premises machines, when your on-premises site goes down, you fail those machines over to Azure. After failover, Azure VMs are created from replicated data.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/Mf9nCXxzfG8" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p class="notice--danger">🛎️ <strong>Important</strong> action checklist table below to be followed <strong>before & after failover</strong>. Read them carefully.</p>
<table>
<thead>
<tr>
<th>Target Failover OS</th>
<th>Stage</th>
<th>Location to perform Actions</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
<tr>
<td><img src="https://imgur.com/Odr4cGss.png" alt="" /> <strong>Windows</strong> Azure VM</td>
<td><strong>Before Failover</strong></td>
<td>On the <strong>On-Premise Machine</strong></td>
<td><strong>For Internet Access</strong>: make sure <strong>RDP is allowed</strong> for <strong>All Profiles</strong>, For <strong>site-to-site VPN access</strong>: make sure <strong>RDP is allowed</strong> for <strong>Domain and Private</strong> network. Make sure <strong>SAN</strong> <a href="https://docs.microsoft.com/en-US/azure/migrate/prepare-for-migration#configure-san-policy"> ( Storage Area Network) </a> policy is set to <strong>Online</strong> and There is no windows update pending on the VM.</td>
</tr>
<tr>
<td><img src="https://imgur.com/Odr4cGss.png" alt="" /> <strong>Windows</strong> Azure VM</td>
<td><strong>After Failover</strong></td>
<td>On the <strong>Azure Virtual Machine</strong></td>
<td><strong>Add a public IP</strong> address, must allow incoming connections to the RDP port.</td>
</tr>
<tr>
<td><img src="https://imgur.com/XnSh8DPs.png" alt="" /> <strong>Linux</strong> Azure VM</td>
<td><strong>Before Failover</strong></td>
<td>On the <strong>On-Premise Machine</strong></td>
<td>Ensure that the Secure Shell Service on the VM is set to start automatically on system boot. Check that firewall rules allow an SSH connection to it.</td>
</tr>
<tr>
<td><img src="https://imgur.com/XnSh8DPs.png" alt="" /> <strong>Linux</strong> Azure VM</td>
<td><strong>After Failover</strong></td>
<td>On the <strong>Azure Virtual Machine</strong></td>
<td><strong>Add a public IP</strong> address, Need to allow incoming connections to the SSH port.</td>
</tr>
</tbody>
</table>
<h3 id="stage-2-reprotect-azure-vms">Stage 2: Reprotect Azure VMs</h3>
<p><img src="https://imgur.com/gvHDxgL.png" alt="" class="full" /></p>
<p>In Azure, you reprotect the Azure VMs so that they start replicating back to the on-premises site. The on-premises VM (if available) is turned off during reprotection, to help ensure data consistency.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/f8LLLYzv4pI" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<h3 id="stage-3-fail-over-from-azure">Stage 3: Fail over from Azure</h3>
<p><img src="https://imgur.com/3jBsHvL.png" alt="" class="full" /></p>
<p>When your on-premises site is running as normal again, you run another failover, this time to fail back Azure VMs to your on-premises site. You can fail back to the original location from which you failed over, or to an alternate location.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/4qG5b6pkXpQ" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<h3 id="stage-4-reprotect-on-premises-machines">Stage 4: Reprotect on-premises machines</h3>
<p>After failing back, again enable replication of the on-premises machines to Azure.</p>
<!-- Courtesy of embedresponsively.com //-->
<div class="responsive-video-container">
<iframe src="https://www.youtube-nocookie.com/embed/o3f83mayYKQ" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</div>
<p><img src="https://imgur.com/XXAYraF.png" alt="" /></p>
<h3 id="overall-replication-steps-model-diagram">Overall Replication Steps Model Diagram</h3>
<p><img src="https://imgur.com/Zuy3Ijz.png" alt="" /></p>
<p class="notice--success">Learn more about <a href="https://docs.microsoft.com/en-us/azure/site-recovery/failover-failback-overview#recovery-stages">BCDR strategy in Azure here</a>.</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariSite Recovery is very critical now a days. In your own Data-center or Server disasters can happen and you must secure your site to recover from them. Use Microsoft Azure to secure you on-premises machine today. Learn the failover and failback stages in Site Recovery on Azure. Learn Business Continuity and Disaster Recovery (BCDR) strategy in Azure.Azure Active Directory Basics2021-12-02T00:00:00+00:002021-12-02T00:00:00+00:00https://www.rupeshtiwari.com/azure-active-directory-basics<blockquote>
<p>Securing your workload and datacenter over the cloud is very challenging. You want your resources to be protected by both machines and users. Azure Active directory helps you to achieve single sign on and provides you centralize identity and access management across your subscriptions. Let’s learn more basic concepts about Azure Active Directory in this article.</p>
</blockquote>
<p>Microsoft provides Azure AD for developers and Microsoft Identity platform to help both programmatic and operational support for single-sign on, Identity and access management.</p>
<h2 id="what-is-azure-active-directory-ad">What is Azure Active Directory (AD)?</h2>
<p><a href="https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/what-is-application-management">Microsoft Azure AD</a> is an Identity and Access Management (IAM) system for the Microsoft cloud. A centralized identity system provides a single place to store user information that can then be used by all applications. These systems have come to be known as Identity and Access Management (IAM) systems. Azure AD performs the authentication using the tenant directory stored in the cloud. Making Azure AD aware of these apps, and how it should handle them, is known as application management. You manage applications on the Enterprise applications page located in the Manage section of the Azure Active Directory portal.</p>
<h2 id="how-does-azure-ad-work-with-apps">How does Azure AD work with apps?</h2>
<p>Azure AD sits in the middle and provides <a href="https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/what-is-application-management">identity management</a> for cloud and on-premises apps. You can integrate on-premises apps using App Proxy.
<img src="https://i.imgur.com/3w439fV.png" alt="" class="full" /></p>
<h2 id="what-is-an-azure-account">What is an Azure account?</h2>
<p>To create or work with an Azure subscription, you must have an Azure account. An Azure account is simply an identity in Azure Active directory (Azure AD).</p>
<h2 id="how-to-take-microsoft-azure-subscription">How to take Microsoft Azure Subscription?</h2>
<p>I want to create an Azure subscription & I do not have an Azure Account. THen if I have account in below directories then I am okay I can create subscriptions:</p>
<ol>
<li>Personal Microsoft Account</li>
<li>School Organization</li>
<li>Work Organization</li>
</ol>
<p>I can create subscriptions in AZure using any account from the above-3 directories.</p>
<h2 id="can-azure-active-directory-work-with-other-organizations">Can Azure Active Directory work with Other Organizations?</h2>
<p>To create or work with an Azure subscription, you must have an Azure account. An Azure account is simply an identity in Azure AD or in a directory, such as a work or school organization, that Azure AD trusts. If you don’t belong to such an organization, you can always create a subscription by using your Microsoft Account, which is trusted by Azure AD.
<img src="https://i.imgur.com/dkAgahS.png" alt="" class="full" /></p>
<h2 id="relationship-between-azure-subscription--azure-active-directory">Relationship between Azure Subscription & Azure Active Directory</h2>
<p>Every Azure subscription has a trust relationship with an Azure AD instance. This means that it trusts that directory to authenticate users, services, and devices. Multiple subscriptions can trust the same directory, but a subscription trusts only one directory.
<img src="https://i.imgur.com/K6yDuoy.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/RgpiTHu.png" alt="" class="full" /></p>
<h2 id="creating-groups-inside-active-directory">Creating Groups inside Active Directory</h2>
<p>As well as defining individual Azure account identities, also called users, you can define groups in Azure AD. Creating user groups is a good way to manage access to resources in a subscription by using role-based access control (RBAC).
<img src="https://i.imgur.com/Kfr6k69.png" alt="" class="full" /></p>
<h2 id="azure-ad-subscription-and-resource-group-relationship">Azure AD Subscription and Resource Group Relationship</h2>
<p>So basically every subscription trust on azure active directory. The users and groups within directory can be assigned role based access control (RBAC) to access resources like storage, compute, network, database and more.</p>
<p><img src="https://i.imgur.com/BzP0hcx.png" alt="" class="full" /></p>
<h2 id="what-is-azure-ad-tenant">What is Azure AD Tenant?</h2>
<p>A tenant represents an organization. A tenant is used by an organization. Example: the tenant used by the Adatum organization who developed HR application.</p>
<p>Azure AD gets created automatically when you create a Microsoft account from Microsoft 365 or Microsoft Intune.</p>
<p><img src="https://i.imgur.com/WajqfHv.png" alt="" /></p>
<h2 id="azure-ad-environments">Azure AD Environments</h2>
<p>A dedicated instance of Azure AD is called “Azure AD Tenant”. You can create a maximum of 20 Azure AD Tenants per Azure subscription. Each Azure AD Tenant is separate from others. Azure AD Tenant has its users or consumes identities. Azure AD Tenant can have resources and they are secured.</p>
<p>Based on the users your application is authenticating you can create 2 types of environments.</p>
<ol>
<li>Work and school or Personal ( outlook.com, live.com )</li>
<li>Social & local (Azure AD B2C )</li>
</ol>
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.youtube.com/playlist?list=PL0azhNeBK66KfW04TZBQWkX62hhnFcb9E">Go Cloud Videos</a></li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariSecuring your workload and datacenter over the cloud is very challenging. You want your resources to be protected by both machines and users. Azure Active directory helps you to achieve single sign on and provides you centralize identity and access management across your subscriptions. Let’s learn more basic concepts about Azure Active Directory in this article.Where to Store Application Data in Azure2021-12-01T00:00:00+00:002021-12-01T00:00:00+00:00https://www.rupeshtiwari.com/where-to-store-application-data-in-azure<blockquote>
<p>Data is very essential and now a days data is everything. So you have application in cloud and want to explore which type of data where to store in Azure. Read this article to learn which Azure store is best for you.</p>
</blockquote>
<h2 id="storing-data-in-azure-options">Storing Data in Azure options</h2>
<ol>
<li><img src="https://imgur.com/CY4p62V.png" alt="" />Azure SQL Database</li>
<li><img src="https://imgur.com/7NE8XB7.png" alt="" />Azure Databases for MySQL</li>
<li><img src="https://imgur.com/0JerP9s.png" alt="" /> Azure Databases for PostgresSQL</li>
<li><img src="https://imgur.com/viVzgo7.png" alt="" /> Azure Databases for MariaDB</li>
<li><img src="https://imgur.com/xr0vqr0.png" alt="" /> Azure Cosmos DB</li>
<li><img src="https://imgur.com/TPnk15t.png" alt="" /> Azure Storage
<ol>
<li><img src="https://imgur.com/0FWw8PM.png" alt="" /> File Storage</li>
<li><img src="https://imgur.com/Wjk68MS.png" alt="" /> Blog Storage</li>
<li><img src="https://imgur.com/MNutzpz.png" alt="" /> Table Storage</li>
<li><img src="https://imgur.com/HsvZyX0.png" alt="" /> Disk Storage</li>
<li><img src="https://imgur.com/iN7LQq0.png" alt="" /> Queue Storage</li>
</ol>
</li>
<li><img src="https://imgur.com/4WSrhP8.png" alt="" /> Azure Synapse Analytics</li>
<li><img src="https://imgur.com/G5KauUF.png" alt="" /> Azure Data Lake Store</li>
</ol>
<h3 id="azure-sql-database">Azure SQL Database</h3>
<p><img src="https://imgur.com/CY4p62V.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure SQL Database</code> can store <strong>Relational Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>SQL Server in the cloud</li>
<li>Managed and Scalable by Azure</li>
<li>Highly available 99.99%</li>
<li>Redundant, data is replicated 3 times in the local data-center and 3 times to another data-center. Data is backed up every 5 minutes and every hour a full backup is made.</li>
<li>Relational data can be used as table rows and columns.</li>
<li>Use it with Microsoft tools
<ul>
<li>SSMS (SQL Server Management Studio), Azure Portal, Visual Studio</li>
</ul>
</li>
<li>Advanced capabilities
<ul>
<li><strong>Geo-replication</strong> (single writable node): which actively replicate data to other geographical region. Other geo replicable databases are read-only.</li>
<li><strong>Automatic tuning</strong>: where Azure SQL analyzes which queries are slow and it will apply required indexes to improve speed.</li>
<li><strong>Threat detection</strong>: it can detect malicious behavior on the database and alert you to take action.</li>
<li><strong>Dynamic Data Masking</strong>: which can automatically mask sensitive data like password, credit card number etc. So un-authorized or un-wanted user can only see masked data only.</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/sql-database/">Azure SQL Database</a></p>
<h3 id="azure-databases-for-mysql">Azure Databases for MySQL</h3>
<p><img src="https://imgur.com/7NE8XB7.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Databases for MySQL</code> can store <strong>Relational Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>MySQL database over the cloud</li>
<li>Managed and Scalable by Azure</li>
<li>Highly available 99.99%</li>
<li>Redundant, data is replicated 3 times in the local data-center and 3 times to another data-center. Data is backed up every 5 minutes and every hour a full backup is made.</li>
<li>Relational data can be used as table rows and columns.</li>
<li>Works well in the LAMP stack
<ul>
<li>Linux, Apache server, MySQL and PHP</li>
</ul>
</li>
<li>Use it with tools for MySQL because most of the Azure tools are not supported for MySQL database.
<ul>
<li><code class="language-plaintext highlighter-rouge">MySQL Workbench</code> and <code class="language-plaintext highlighter-rouge">SQLyog</code>, drivers for PHP, Java, Node.js and .NET.</li>
<li>Can integrate with<a href="https://www.rupeshtiwari.com/what-should-you-use-for-running-applications-in-azure/#azure-app-services-web-apps"> Azure App Service</a></li>
</ul>
</li>
</ul>
<p class="notice--success"><code class="language-plaintext highlighter-rouge">MySQL</code> is <code class="language-plaintext highlighter-rouge">open-source</code> database. Learn more about <a href="https://azure.microsoft.com/en-us/services/mysql/#features">Azure Databases for MySQL</a></p>
<h3 id="azure-databases-for-postgresql">Azure Databases for PostgreSQL</h3>
<p><img src="https://imgur.com/0JerP9s.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Databases for PostgreSQL</code> can store <strong>Relational Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>PostgresSQL in the cloud</li>
<li>Managed and Scalable by Azure</li>
<li>Highly available 99.99%</li>
<li>Redundant, data is replicated 3 times in the local data-center and 3 times to another data-center. Data is backed up every 5 minutes and every hour a full backup is made.</li>
<li>Relational data can be used as table rows and columns.
<img src="https://imgur.com/EWnJywn.png" alt="" /></li>
<li>Flexibility
<ul>
<li>Write functions in several languages (Ruby on Rails, Python with Django, Java with Spring Boot, and Node.js)</li>
<li>Use PostgresSQL extensions (PLV8, and PostGIS)</li>
</ul>
</li>
<li>Use it with tools for PostgreSQL
<ul>
<li><code class="language-plaintext highlighter-rouge">pgAdmin</code></li>
<li><code class="language-plaintext highlighter-rouge">Hyperscale (Citus)</code> extension to scale to hundreds of nodes with no application rewrites.</li>
</ul>
</li>
</ul>
<p class="notice--success"><code class="language-plaintext highlighter-rouge">PostgreSQL</code> is <code class="language-plaintext highlighter-rouge">open-source</code> database. Learn more about <a href="https://azure.microsoft.com/en-us/services/postgresql">Azure Databases for PostgreSQL</a></p>
<h3 id="azure-databases-for-mariadb">Azure Databases for MariaDB</h3>
<p><img src="https://imgur.com/viVzgo7.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Databases for PostgreSQL</code> can store <strong>Relational Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>MariaDB in the cloud</li>
<li>Managed and Scalable by Azure</li>
<li>Highly available 99.99%</li>
<li>Redundant, data is replicated 3 times in the local data-center and 3 times to another data-center. Data is backed up every 5 minutes and every hour a full backup is made.</li>
<li>Relational data can be used as table rows and columns.</li>
<li>Supports different storage engines that allows to store data in other format like
<ul>
<li><code class="language-plaintext highlighter-rouge">Apache Cassandra, CSV, ColumnStore</code></li>
</ul>
</li>
<li>Use it with the tools for MariaDB
<ul>
<li><code class="language-plaintext highlighter-rouge">Database Workbench</code> and <code class="language-plaintext highlighter-rouge">SQLyog</code></li>
</ul>
</li>
</ul>
<p class="notice--success">Did you know <strong>MariaDB</strong> is forked out from <strong>MySQL</strong> and <code class="language-plaintext highlighter-rouge">open source</code>? Learn more about <a href="https://docs.microsoft.com/en-us/azure/mariadb/#:~:text=Azure%20Database%20for%20MariaDB%20is,predictable%20performance%20and%20dynamic%20scalability.">Azure Databases for MariaDB</a></p>
<h3 id="azure-cosmos-db">Azure Cosmos DB</h3>
<p><img src="https://imgur.com/xr0vqr0.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Cosmos DB</code> can store <strong>Document Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>Distributed, massive scale database</li>
<li>Use it with several APIs to work with your data in many different ways and they abstract the way that Azure Cosmos DB stores and handles the data.
<ul>
<li>DocumentDB API</li>
<li>MongoDB API</li>
<li>Table API</li>
<li>Graph API</li>
<li>Apache Cassandra API</li>
</ul>
</li>
<li>Since it is <code class="language-plaintext highlighter-rouge">noSQL</code> you do not need to predefine the schema</li>
<li><strong>Automatic indexing</strong>: Cosmos DB will automatically create and apply and manages indexes required on data. You can customize the indexing policy if you want.</li>
<li><strong>Geo-replication</strong> (single writable node): which actively replicate data to other geographical region. Other geo replicable databases are read-only.</li>
<li>Unique SLA is offered by CosmosDB
<ul>
<li>SLA <code class="language-plaintext highlighter-rouge">99.99%</code> availability</li>
<li>SLA <code class="language-plaintext highlighter-rouge">99.99%</code> availability for <code class="language-plaintext highlighter-rouge">read</code> operation</li>
<li>Latency on <code class="language-plaintext highlighter-rouge">read</code> <code class="language-plaintext highlighter-rouge">< 10ms</code>, <code class="language-plaintext highlighter-rouge">writes < 15ms</code>
This latency counts from the Cosmos DB to other Azure services.</li>
</ul>
</li>
</ul>
<p class="notice--success"><code class="language-plaintext highlighter-rouge">Apache Cassandra </code>is a <code class="language-plaintext highlighter-rouge">free</code> and <code class="language-plaintext highlighter-rouge">open-source</code>, distributed, wide-column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Learn more about <a href="https://azure.microsoft.com/en-us/services/cosmos-db/">Azure Cosmos DB</a></p>
<h3 id="azure-storage">Azure Storage</h3>
<p><img src="https://imgur.com/TPnk15t.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Storage</code> can store <strong>Unstructured/Document Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>Massive scale, multiple-purpose storage.</li>
<li>Multiple models in a storage account
<ul>
<li>File Storage</li>
<li>Blob Storage</li>
<li>Queue Storage</li>
<li>Table Storage</li>
<li>Disk Storage</li>
</ul>
</li>
</ul>
<p>All storage come with overall features and capabilities that Azure Storage provides like</p>
<ul>
<li><strong>Shared Access Signatures</strong>: provides temporary access to some operations on storage type like Blob storage. This is useful for implementing the <code class="language-plaintext highlighter-rouge">valet key</code> pattern.</li>
<li><code class="language-plaintext highlighter-rouge">Firewalls</code> and <code class="language-plaintext highlighter-rouge">Virtual Network</code> using this you can increase security to your Azure storage data.</li>
<li><code class="language-plaintext highlighter-rouge">Data encryption</code> at rest. Azure storage will encrypt and decrypt the data for you out of the box.</li>
<li>Storage replication
<ul>
<li><strong>Locally-redundant</strong> : 3 copies of the data stored in the local data-center</li>
<li><strong>Geo-redundant</strong>: In addition to Locally-redundant, it will store 3 copies of data in the other geographical data-center</li>
<li><strong>Read-access-geo-redundant</strong>: You can also create readable Geo-redundant also.</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/">Azure Storage</a></p>
<h3 id="azure-file-storage">Azure File Storage</h3>
<p><img src="https://imgur.com/0FWw8PM.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure File Storage</code> can store <strong>Unstructured Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>Really useful for <code class="language-plaintext highlighter-rouge">Lift and Shift</code> files to the cloud. Useful when you want to keep the file available across the applications.</li>
<li>File storage based on <code class="language-plaintext highlighter-rouge">SMB</code>, <code class="language-plaintext highlighter-rouge">NSF</code></li>
<li>Mount an <code class="language-plaintext highlighter-rouge">Azure File</code> share in
<ul>
<li>Windows</li>
<li>MacOS</li>
<li>Linux</li>
</ul>
</li>
<li>You get all Azure Storage features
<ul>
<li>Shared Access Signatures</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/files/">Azure File Storage</a></p>
<h3 id="azure-blob-storage">Azure Blob Storage</h3>
<p><img src="https://imgur.com/Wjk68MS.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Blob Storage</code> can store <strong>Unstructured Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>Store <code class="language-plaintext highlighter-rouge">unstructured</code> data in the cloud like video, audio etc.</li>
<li>Multiple types of blob can be stored
<ul>
<li><strong>Block</strong>: blobs are optimized for <code class="language-plaintext highlighter-rouge">efficient upload of large file</code>.</li>
<li><strong>Page</strong>: blobs are optimized for <code class="language-plaintext highlighter-rouge">random read and write access</code> like <code class="language-plaintext highlighter-rouge">VHD</code> files for virtual machine image.</li>
<li><strong>Append</strong>: these blobs can only be added onto existing blob and the blob can’t be changed. Like adding logs on log blog.</li>
</ul>
</li>
<li>
<p>Multiple storage tiers</p>
<ul>
<li><strong>Premium</strong>: access blobs anytime with <code class="language-plaintext highlighter-rouge">great SSD performance</code> and <code class="language-plaintext highlighter-rouge">very expensive</code></li>
<li><strong>Hot</strong>: access blobs anytime with <code class="language-plaintext highlighter-rouge">great performance</code> and <code class="language-plaintext highlighter-rouge">expensive</code></li>
<li><strong>Cool</strong>: <code class="language-plaintext highlighter-rouge">cheaper</code> but take more time to fetch blob.</li>
<li><strong>Archive</strong>: store offline data, <code class="language-plaintext highlighter-rouge">cheaper storage</code> and <code class="language-plaintext highlighter-rouge">costly retrieval</code></li>
</ul>
</li>
<li>You get all Azure Storage features
<ul>
<li>Firewalls and Virtual Network</li>
<li>Shared Access Signatures</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/blobs/">Azure Blob Storage</a></p>
<h3 id="azure-table-storage">Azure Table Storage</h3>
<p><img src="https://imgur.com/MNutzpz.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Table Storage</code> can store <strong>Unstructured Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li>A <strong>NoSQL key-value store for rapid development using massive semi-structured datasets</strong>.</li>
<li><strong>Flexible data schema</strong>: the table schema is flexible means one row might have 5 the other can have 23 columns.</li>
<li>You get all Azure Storage features
<ul>
<li>Firewalls and Virtual Network</li>
<li>Shared Access Signatures</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/tables/">Azure Blob Storage</a></p>
<h3 id="azure-disk-storage">Azure Disk Storage</h3>
<p><img src="https://imgur.com/HsvZyX0.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Disk Storage</code> can store <strong>Unstructured Data</strong>. It is optimized for <a href="#online-transaction-processing-oltp-datastores"><strong>OLTP (Online Transactional Processing)</strong></a>.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">High-performance</code>, highly <code class="language-plaintext highlighter-rouge">durable</code> <code class="language-plaintext highlighter-rouge">block storage for Azure Virtual Machines</code>.</li>
<li>Part of the <code class="language-plaintext highlighter-rouge">premium</code> <code class="language-plaintext highlighter-rouge">pricing</code> tier of Azure storage only.</li>
<li>Perfect for use in Virtual Machines.</li>
<li>You can use this to <code class="language-plaintext highlighter-rouge">lift and shift the existing applications</code> like a Dynamic CRM server to a VM in the cloud. And you get availability and robustness of the cloud.</li>
<li>You get all Azure Storage features
<ul>
<li>Firewalls and Virtual Network</li>
<li>Shared Access Signatures</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/disks/">Azure Disk Storage</a></p>
<h3 id="azure-synapse-analytics">Azure Synapse Analytics</h3>
<p><img src="https://imgur.com/4WSrhP8.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Synapse Analytics</code> can store <strong>Analytics Data</strong>. It is optimized for <a href="#online-analytical-processing-olap-datastores"><strong>OLAP (Online Analytical Processing)</strong></a>.</p>
<ul>
<li><strong>For prepared, structured data for reporting</strong>: Azure Synapse Analytics is perfect for when you want to store a <code class="language-plaintext highlighter-rouge">prepared</code> and <code class="language-plaintext highlighter-rouge">structured</code> data specifically for reporting purposes in so-called <code class="language-plaintext highlighter-rouge">data marts</code>.</li>
<li><strong>Relational Data</strong>: You only stored relational data which is cleaned and ready to be used for the users that use a reporting tool like <code class="language-plaintext highlighter-rouge">Power BI</code>. It is <code class="language-plaintext highlighter-rouge">similar</code> to <code class="language-plaintext highlighter-rouge">Azure SQL database</code> optimized for reporting purpose.</li>
<li>Large amounts <a href="https://docs.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/ sql-data-warehouse-service-capacity-limits">of data up to 1PB ~ 1 million GB</a> can be stored in it.</li>
<li><strong>Data encryption at rest</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">Transparent Data encryption</code> just like Azure SQL database.</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/synapse-analytics/">Azure Synapse Analytics</a></p>
<h3 id="azure-data-lake-store">Azure Data Lake Store</h3>
<p><img src="https://imgur.com/G5KauUF.png" alt="" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Data Lake Store</code> can store <strong>Analytics Data</strong>. It is optimized for <a href="#online-analytical-processing-olap-datastores"><strong>OLAP (Online Analytical Processing)</strong></a>.</p>
<ul>
<li><strong>For structured and unstructured data for reporting</strong>: in it’s native format.</li>
<li><strong>Data in native format</strong>: so you don’t need to clean and prepare before you put the data in Azure Data Lake Store</li>
<li><strong>No Schema definition</strong>: you don’t have to create a data schema up front, which you must do in Azure Synapse Analytics. You can use Azure Data Lake to just store the big data and further move them in to Azure Synapse by cleaning and preparing it.</li>
<li><strong>Optimized for reporting purposes</strong>: high performance reporting and analytics workloads are available.</li>
<li><strong>No File or Datastore size limits</strong>: You can store large amount of unlimited data. The only limit is your 💵 budget :)</li>
<li><strong>Data encryption at rest</strong>: out of the box you get this.</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/storage/data-lake-storage/">Azure Data Lake Store</a></p>
<h2 id="online-transaction-processing-oltp-datastores">Online Transaction Processing (OLTP) DataStores</h2>
<p>Online Transactional Processing (PLTP) datastores are <code class="language-plaintext highlighter-rouge">optimized</code> for the <code class="language-plaintext highlighter-rouge">transactional processing</code> for <code class="language-plaintext highlighter-rouge">create, read, update and delete</code> operations. These are typically used as data storage for live <code class="language-plaintext highlighter-rouge">applications like website</code> or <code class="language-plaintext highlighter-rouge">mobile apps</code>.</p>
<p><strong>List of Data Storage supporting OLTP</strong></p>
<ol>
<li><img src="https://imgur.com/CY4p62V.png" alt="" />Azure SQL Database</li>
<li><img src="https://imgur.com/7NE8XB7.png" alt="" />Azure Databases for MySQL</li>
<li><img src="https://imgur.com/0JerP9s.png" alt="" /> Azure Databases for PostgresSQL</li>
<li><img src="https://imgur.com/viVzgo7.png" alt="" /> Azure Databases for MariaDB</li>
<li><img src="https://imgur.com/xr0vqr0.png" alt="" /> Azure Cosmos DB</li>
<li><img src="https://imgur.com/TPnk15t.png" alt="" /> Azure Storage
<ol>
<li><img src="https://imgur.com/0FWw8PM.png" alt="" /> File Storage</li>
<li><img src="https://imgur.com/Wjk68MS.png" alt="" /> Blog Storage</li>
<li><img src="https://imgur.com/MNutzpz.png" alt="" /> Table Storage</li>
<li><img src="https://imgur.com/HsvZyX0.png" alt="" /> Disk Storage</li>
<li><img src="https://imgur.com/iN7LQq0.png" alt="" /> Queue Storage</li>
</ol>
</li>
</ol>
<h2 id="online-analytical-processing-olap-datastores">Online Analytical Processing (OLAP) DataStores</h2>
<p>Online Analytical Processing (OLAP) datastores are <code class="language-plaintext highlighter-rouge">optimized</code> for <code class="language-plaintext highlighter-rouge">analytical processing</code> like crunching of data for reporting purposes.</p>
<p><strong>Below are the list of OLAP datastores</strong></p>
<ol>
<li><img src="https://imgur.com/4WSrhP8.png" alt="" /> Azure Synapse Analytics</li>
<li><img src="https://imgur.com/G5KauUF.png" alt="" /> Azure Data Lake Store</li>
</ol>
<h2 id="comparing-the-datastores">Comparing the DataStores</h2>
<p>Let’s compare datastores as per OLAP and OLTP categories.</p>
<h3 id="compare-data-stores-for-oltp">Compare Data Stores for OLTP</h3>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
The main difference among <code class="language-plaintext highlighter-rouge">Azure SQL, MySQL, PSQL and MariaDB</code> is the <code class="language-plaintext highlighter-rouge">ecosystem</code> they are in and the skillset that you need for them. If you are comfortable on <code class="language-plaintext highlighter-rouge">LAMP</code> stack and working with <code class="language-plaintext highlighter-rouge">MySQL</code> then Azure Databases for <code class="language-plaintext highlighter-rouge">MySQL</code> or <code class="language-plaintext highlighter-rouge">MariaDB</code> is good for you. If you are comfortable working in <code class="language-plaintext highlighter-rouge">pgAdmin</code> or similar tool and you are not using <code class="language-plaintext highlighter-rouge">T-SQL</code> to manage database then use <code class="language-plaintext highlighter-rouge">Azure PostgreSQL</code>.</p>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/CY4p62V.png" alt="" /> SQL</th>
<th><img src="https://imgur.com/7NE8XB7.png" alt="" /> MySQL</th>
<th><img src="https://imgur.com/0JerP9s.png" alt="" /> PSQL</th>
<th><img src="https://imgur.com/viVzgo7.png" alt="" /> MariaDB</th>
<th><img src="https://imgur.com/xr0vqr0.png" alt="" />Cosmos DB</th>
<th><img src="https://imgur.com/TPnk15t.png" alt="" /> Storage</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Relational data</strong></td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Semi- or non-relational data data</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Can use Microsoft tools to manage</strong></td>
<td>✔️</td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Advanced querying capabilities</strong></td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>Use multiple APIs to access data</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<h3 id="compare-azure-storage-type-services">Compare Azure Storage Type Services</h3>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<strong>All Azure storage</strong> types have the same Azure Storage Features like <strong>Shared Access Signatures</strong> and <strong>data encryption at rest</strong>.</p>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/0FWw8PM.png" alt="" /> File</th>
<th><img src="https://imgur.com/Wjk68MS.png" alt="" /> Blob</th>
<th><img src="https://imgur.com/MNutzpz.png" alt="" /> Table</th>
<th><img src="https://imgur.com/HsvZyX0.png" alt="" /> Disk</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Lift and shift files to the cloud</strong></td>
<td>✔️</td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Lift and shift applications to the cloud</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Store (large) unstructured data</strong></td>
<td> </td>
<td>✔️</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Store (small) semi-structured data</strong></td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<h3 id="compare-data-stores-for-olap">Compare Data Stores for OLAP</h3>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
The <strong>Data Lake Store</strong> stores the relational structured data in <code class="language-plaintext highlighter-rouge">text</code> file or <code class="language-plaintext highlighter-rouge">hierarchical</code> <code class="language-plaintext highlighter-rouge">folders</code>. You can also define the schema in Azure Data Lake store but this is not compulsory like Azure Synapse Analytics store.</p>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/4WSrhP8.png" alt="" /> Azure Synapse Analytics</th>
<th><img src="https://imgur.com/G5KauUF.png" alt="" /> Azure Data Lake Store</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Relational data</strong></td>
<td>✔️</td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Non-relational data like documents</strong></td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Requires you need to define data schema</strong></td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>No file- or database size limits</strong></td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Use when you know which questions you want the data to answer</strong></td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariData is very essential and now a days data is everything. So you have application in cloud and want to explore which type of data where to store in Azure. Read this article to learn which Azure store is best for you.Azure solution for big-data transfer2021-11-29T00:00:00+00:002021-11-29T00:00:00+00:00https://www.rupeshtiwari.com/azure-solution-for-big-data-transfer<blockquote>
<p>Are you planning to migrate your on-premise data to the cloud using Azure data transfer solutions?</p>
</blockquote>
<p>At a high level there are only 2 solutions to transfer your on-premise data to the cloud.</p>
<ol>
<li>Offline Data Transfer</li>
<li>Online Data Transfer</li>
</ol>
<h2 id="offline-data-transfer">Offline data transfer</h2>
<p>Cloud providers will ship you a secure specialized device or disk based on the volume of data. You copy data to that device, apply a secret pin and then send it back to Azure where the data is uploaded and you can verify your Azure Storage account for the uploaded data. Sometimes if data is not that big then you can purchase a disk of your own and ship it to azure. Azure has a Databox family to help you transfer your data offline. Data Box Disk, Data Box, Data Box Heavy, and import/export (use your own disk).</p>
<p>The data box is normally 8-TB BitLocker-encrypted SSD upto 35 TB capacity. If you need more data then you order more of them. Yes you can put 2 or 3 disks of 35TB each and copy parallels data on them using SMB protocol and tools like Robocopy. The disk will have predefined folders like PageBlob, BlockBlob, AzureFile, or ManagedDisk.</p>
<p>File/Folder restrictions while copying:</p>
<ul>
<li>Blob blocks file size must not exceed 4.75 tebibytes (1 TiB = 240 bytes )</li>
<li>Page blocks file size must not exceed 8 tebibytes (TiB)</li>
<li>
<p>Azure Files file size must not exceed 1 tebibytes (TiB)</p>
</li>
<li>Subfolder names should be lowercase, from 3 to 63 characters, and consist only of letters, numbers, and hyphens. Consecutive hyphens aren’t allowed.</li>
<li>File names for Azure Files shouldn’t exceed 255 characters.</li>
</ul>
<p>What to copy where?</p>
<ul>
<li>ManagedDisk folder for VHD (Virtual Hard Disk) files.</li>
<li>PageBlob folder to save VHDX (successor of Virtual Hard Disk) files.</li>
<li>BlockBlob folder to save images, documents etc.</li>
<li>AzureFile folder to save folder and files</li>
</ul>
<p>The only bad part of this option is it takes time to do this way of data transfer. You have to create an order, wait around 10 days for the device to come, copy, ship it back to Azure, then upload data and finally erase the data as per the National Institute of Standard and Technology (NIST) guidelines.</p>
<p><img src="https://lh4.googleusercontent.com/T28XMm5nJkyajL7-9MuUjDB7XejV9EglIxvcphhcwRmD2EVF5pzenuybNnfzB7HzTKv8aeJsTEDbRpmMwjlktCcTvhE8UAfj00R2Ya94IJNOZ-E-b3T7bU5EMtlWEttrxYNqw_NJ=s0" alt="" /></p>
<p>Note: For Data Box Heavy, make sure that it will fit through the entrance of your premises, and that you have at least two people to handle it. It’s a massive device that weighs around 500 lbs and arrives on wheels 😃</p>
<h2 id="online-data-transfer">Online data transfer </h2>
<p>You use your internet bandwidth to securely transfer data to azure cloud over the network. You can use Azure Applications to transfer data like Azure Storage Explorer or Azure web portal. You can even write your own powershell or node.js .net java etc. script to transfer data. You can use the Azure data factory to create a pipeline to set up dedicated data transfer. Finally if you have some analytics to be done before you send data over cloud like filter PII data or any other custom filter, logic and want to optimise transfer by cache frequent data etc. then use Azure Stack Edge. You need a physical device, the Azure Stack Edge and virtual device the Data Box Gateway. Azure ships you Azure Stack Edge device, you install it on-premise permanently and connect to Azure over your network.</p>
<p><img src="https://lh4.googleusercontent.com/f0erDWSp3rKqmhxokCTuDVageaobRC4ZRaCAKq2YHoDO2dRoZShLGocRjMTl4haB8e5bGO4rZX18Klgvn4RviYqn9BgPNOu4ddgbmXNK2GyGFT9a7D3oEKHbhxHeFnPXEDX0QHDF=s0" alt="" /></p>
<h2 id="how-to-decide-which-data-transfer-solution-to-select">How to decide which data transfer solution to select?</h2>
<p>You should answer below 3 questions to help yourself to decide which solution is good for your data transfer to the cloud.</p>
<ul>
<li>What is the size/volume of the data?</li>
<li>How much network bandwidth is available in my on-premise environment?</li>
<li>How often do I want to transfer data, is it one-time or continuous data transfer?</li>
</ul>
<h2 id="general-cloud-data-transfer-scenarios">General Cloud Data Transfer Scenarios </h2>
<p>Let’s discuss major scenarios of data transfer to the cloud.</p>
<ul>
<li>Do you want to transfer a large volume of data with low network bandwidth or is it expensive?</li>
<li>Do you want to transfer a large volume of data with moderate to high network bandwidth?</li>
<li>Do you want to transfer a small volume of data with moderate network bandwidth?</li>
<li>Do you want to transfer a point-in-time data transfer at regular intervals?</li>
<li>Do you want to transfer an on-going data transfer at regular intervals?</li>
</ul>
<h2 id="data-transfer-for-large-datasets-with-low-or-no-network-bandwidth">Data transfer for large datasets with low or no network bandwidth</h2>
<p>Large amounts of data means few TBs or PBs of data you may not want to send over wire because of low internet speed (100 Mbps - 1 Gbps) or security policy which does not allow you to transfer data over wire. In this scenario, I prefer a physical device to do a one-time bulk data transfer. Choose from Data Box Disk, Data Box, Data Box Heavy devices which are supplied by Microsoft, or Import/Export using your own disks.</p>
<p>If you are still confused should you use physical device or not then use below table to decide</p>
<p><img src="https://lh3.googleusercontent.com/DF7h0tCmu6MXxgjnPbwH_Yl89dcvQA9NPehjKzbk_1TS-eUkwMwfeJyh75wcXjVALZlDq7uBmIqA21bFXci3jZ3ZdUEBuM6CkgMU-lN_D96-3WH6yQEIaz75Azz_88LA_zEJJCtD=s0" alt="" /></p>
<h2 id="data-transfer-for-large-datasets-with-high-network-bandwidth">Data transfer for large datasets with high network bandwidth</h2>
<p>If you have high network bandwidth (1 Gbps - 50 Gbps) and you rely on cloud security setup then prefer data transfer via network bandwidth only.</p>
<p>Powershell AzCopy CLI is a good choice to spin-up parallel processes to transfer data over the network to the Azure cloud. It will resume transferring if the data transfer is interrupted. However, it needs technical knowledge.</p>
<p>Rest API exposed by Azure Storage is also a good choice if you have your development team to create UI for you to transfer data.</p>
<p>Azure Data Box family for online transfer: If you want to do live streaming of data to the cloud and also you want to pre-process your data then prefer using Azure Data Box Edge physical device with Data Box Gateway virtual device together.</p>
<p>Azure Data Factory: Use Data Factory to regularly transfer files between several Azure services, on-premises, or a combination of the two. with Data Factory, you can create and schedule data-driven workflows (called pipelines) that ingest data from disparate data stores and automate data movement and data transformation.</p>
<p><a href="https://docs.microsoft.com/en-us/azure/storage/common/storage-solution-large-dataset-moderate-high-network">Refer this article</a> to learn more about comparisons on which tool you use.</p>
<h2 id="data-transfer-for-small-datasets-with-low-to-moderate-network-bandwidth">Data transfer for small datasets with low to moderate network bandwidth</h2>
<p>If you have network speed around ( 50 Mbps - 1 Gbps ) and data is low then go for an online data transfer solution that I described in the above scenario.</p>
<h2 id="solutions-for-periodic-data-transfer">Solutions for periodic data transfer</h2>
<p>If you want to transfer data at specific calendar time frequently like point-in-time once in a week/month then based on your internet speed, policies and data size you can choose either offline or online data transfer.</p>
<p>If you want to transfer data from on-premise to cloud on a regular basis like every day or continuous transfer then prefer online data transfer option only. You can use</p>
<ul>
<li>Object replication - In Azure you can use object replication to sync containers between multiple azure storage accounts.</li>
<li>Azure data factory - You can orchestrate data-driven workflows that can combine both on-prem and azure data and provide automatic data transmission to the cloud.</li>
<li>Also try Azure Data Box Family for online transfer - this option is for excellent performance and data filtering.</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li>Moderate Network Bandwidth is 100 Mbps - 1 Gbps</li>
<li>High Network Bandwidth is 1 Gbps - 50 Gbps</li>
<li>1 GB = 109 Bytes</li>
<li>1 TB = 1012 Bytes</li>
<li>1 PB (Petabyte) = 1015 Bytes</li>
<li><a href="https://docs.microsoft.com/en-us/azure/storage/common/storage-choose-data-transfer-solution">https://docs.microsoft.com/en-us/azure/storage/common/storage-choose-data-transfer-solution</a></li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariAre you planning to migrate your on-premise data to the cloud using Azure data transfer solutions?What to use to Run applications in Azure2021-11-27T00:00:00+00:002021-11-27T00:00:00+00:00https://www.rupeshtiwari.com/what-should-you-use-for-running-applications-in-azure<blockquote>
<p>Learn what service from Azure you should use while creating your application in Azure. This article has quick short answer for when to use what in Azure service? Get <a href="https://azure.microsoft.com/en-us/resources/designing-distributed-systems/">free e-book on Design Distributed Systems</a>.
It is difficult to compare options available in Azure. The comparison with the features of the options provides you tools to choose the right service with your requirements.</p>
</blockquote>
<h2 id="running-application-in-azure-options">Running Application in Azure Options</h2>
<p>You can use various services from Azure to run your applications. It depends what exactly is your requirements. Below are the services available from Azure.</p>
<ol>
<li><img src="https://imgur.com/YPcnULJ.png" alt="" /> Azure Virtual Machines. (VMs)</li>
<li><img src="https://imgur.com/ukowMIs.png" alt="" /> Azure Kubernetes Services (AKS)</li>
<li><img src="https://imgur.com/5IrDqJM.png" alt="" /> Container instances</li>
<li><img src="https://imgur.com/qwLOnPT.png" alt="" />Azure Web App for Containers</li>
<li><img src="https://imgur.com/3HK1rtT.png" alt="" />Azure Batch</li>
<li><img src="https://imgur.com/cdq4W3I.png" alt="" />Azure Service Fabric (Mesh)</li>
<li><img src="https://imgur.com/eIRJkJj.png" alt="" />Cloud ServicesX</li>
<li><img src="https://imgur.com/iH0hVvs.png" alt="" />Azure App Services Web Apps</li>
<li><img src="https://imgur.com/OcHd8Qi.png" alt="" /> Azure App Services WebJobs</li>
<li><img src="https://imgur.com/SRk7YqX.png" alt="" />Azure App Services Mobile Apps</li>
<li><img src="https://imgur.com/j1Y4A4B.png" alt="" /> Function Apps</li>
<li><img src="https://imgur.com/z0fI8LX.png" alt="" /> Azure Logic Apps</li>
</ol>
<h3 id="virtual-machines-vms">Virtual Machines (VMs)</h3>
<p><img src="https://imgur.com/lVMTAlR.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Virtual Machines</code> is under <strong>Infrastructure as a Service (IaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Managed Virtual Machine Service in Cloud</li>
<li>Starts and Stops in <code class="language-plaintext highlighter-rouge">minutes</code></li>
<li>Many OS images are available.
<ul>
<li>You can choose your own OS image (Linux Windows). Explore the full list of <a href="https://azure.microsoft.com/en-us/pricing/details/virtual-machines/series/">Azure VM types</a></li>
<li>You can bring your own OS /Software License ( same money )</li>
</ul>
</li>
<li>Many Sizes to choose from (CPU, Memory, Disk Type)
<ul>
<li><code class="language-plaintext highlighter-rouge">General purpose</code></li>
<li><code class="language-plaintext highlighter-rouge">GPU</code> (for Intense work like video rendering etc.)</li>
</ul>
</li>
<li>Configuration options
<ul>
<li>Auto shutdown, Adding Disks, Adding VM to Networking</li>
</ul>
</li>
<li>SLA <code class="language-plaintext highlighter-rouge">99.95%</code> (Only if you run multiple instances)</li>
</ul>
<p class="notice--success">Learn <a href="https://docs.microsoft.com/en-us/azure/virtual-machines/">Getting Started with Azure Virtual Machine</a>. To get guarantied 99.95% SLA you must need 2 or more VMs.</p>
<h3 id="azure-kubernetes-services-aks">Azure Kubernetes Services (AKS)</h3>
<p><img src="https://imgur.com/H50OFsZ.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Kubernetes Service</code> is under <strong>Infrastructure as a Service (IaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Run <code class="language-plaintext highlighter-rouge">multiple</code> containers that makeup your applications</li>
<li>Starts and stops in <code class="language-plaintext highlighter-rouge">seconds</code></li>
<li>Managed version of the Kubernetes container <code class="language-plaintext highlighter-rouge">orchestrator</code>. It takes care of provisioning or de-provisioning the containers. It scales up and down and monitor containers.</li>
<li>Runs <code class="language-plaintext highlighter-rouge">Linux</code> and <code class="language-plaintext highlighter-rouge">Windows</code> Containers</li>
<li>You instantiate the container with images from
<ul>
<li><code class="language-plaintext highlighter-rouge">Docker Hub</code> or</li>
<li><code class="language-plaintext highlighter-rouge">Azure Container Registry</code>, here you can host your own images or upload from somewhere.</li>
</ul>
</li>
<li>AKS runs in Azure on a cluster of VM so the availability is same as Azure VM.</li>
</ul>
<p class="notice--success">Containers are <code class="language-plaintext highlighter-rouge">light weight</code> compare to VM. They starts and stops in <code class="language-plaintext highlighter-rouge">Seconds</code> however VMs in <code class="language-plaintext highlighter-rouge">Minutes</code>. Learn more about <a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">AKS</a>.</p>
<h3 id="container-instances">Container Instances</h3>
<p><img src="https://imgur.com/9aKMeiF.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Container Instances</code> is under <strong>Infrastructure as a Service (IaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Run <code class="language-plaintext highlighter-rouge">single</code> containers that makeup your applications</li>
<li>Starts and stops in <code class="language-plaintext highlighter-rouge">seconds</code></li>
<li><code class="language-plaintext highlighter-rouge">No</code> container <code class="language-plaintext highlighter-rouge">orchestrator</code> needed
<ul>
<li>Can work with orchestrators</li>
</ul>
</li>
<li>Runs <code class="language-plaintext highlighter-rouge">Linux</code> and <code class="language-plaintext highlighter-rouge">Windows</code> Containers</li>
<li>You instantiate the container with images from
<ul>
<li><code class="language-plaintext highlighter-rouge">Docker Hub</code> or</li>
<li><code class="language-plaintext highlighter-rouge">Azure Container Registry</code>, here you can host your own images or upload from somewhere.</li>
</ul>
</li>
<li>AKS runs in Azure on a cluster of VM so the availability is same as Azure VM.</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/container-instances/">Azure Container Instances</a>.</p>
<h3 id="azure-web-app-for-containers">Azure Web App for Containers</h3>
<p><img src="https://imgur.com/ciwt04O.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Web App for Containers</code> is under <strong>Infrastructure as a Service (IaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>You can run container in Azure Web App for Container as Service. This allows you to run container on App Services.</li>
<li>Starts and stops in <code class="language-plaintext highlighter-rouge">seconds</code></li>
<li><code class="language-plaintext highlighter-rouge">No</code> container <code class="language-plaintext highlighter-rouge">orchestrator</code> needed</li>
<li>Runs <code class="language-plaintext highlighter-rouge">Linux</code> and <code class="language-plaintext highlighter-rouge">Windows</code> Containers</li>
<li>Use App Service Features
<ul>
<li>Easy to setup <code class="language-plaintext highlighter-rouge">Continuous Deployments</code> from source control</li>
<li>Use <code class="language-plaintext highlighter-rouge">Deployment Slots</code> which allows you deployment with no <code class="language-plaintext highlighter-rouge">downtime</code></li>
<li><code class="language-plaintext highlighter-rouge">Scale</code> and automatic Scale by adding rules.</li>
</ul>
</li>
<li>You instantiate the container with images from
<ul>
<li><code class="language-plaintext highlighter-rouge">Docker Hub</code> or</li>
<li><code class="language-plaintext highlighter-rouge">Azure Container Registry</code>, here you can host your own images or upload from somewhere.</li>
</ul>
</li>
</ul>
<h3 id="azure-batch">Azure Batch</h3>
<p><img src="https://imgur.com/q3vIFLU.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Batch</code> is under <strong>Infrastructure as a Service (IaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Allows you to run your job across many <code class="language-plaintext highlighter-rouge">Virtual Machines</code> at <code class="language-plaintext highlighter-rouge">massive</code> scale (thousands of VMs). You can use it to run a calculating job that needs lots of compute power or rendering video files.</li>
<li>Orchestrates running your job in parallel
<ul>
<li><code class="language-plaintext highlighter-rouge">Scaling</code> across machines</li>
<li><code class="language-plaintext highlighter-rouge">Stages data</code> for your job so you can insert data into your job.</li>
<li>Create <code class="language-plaintext highlighter-rouge">job pipelines</code> with tasks so you can determine what steps need to be done with the output.</li>
</ul>
</li>
<li>Runs your job on <code class="language-plaintext highlighter-rouge">Linux</code> and <code class="language-plaintext highlighter-rouge">Windows</code> based VMs</li>
<li>Can <code class="language-plaintext highlighter-rouge">run containers</code>. So you can run your application in container in Azure Batch. It will run it on as many VMs as needed.</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/batch/">Azure Batch</a></p>
<h3 id="azure-service-fabric-mesh">Azure Service Fabric (Mesh)</h3>
<p><img src="https://imgur.com/EDhiF4b.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Service Fabric (Mesh)</code> is under <strong>Infrastructure as a Service (IaaS)</strong> as well as <strong>Platform as a Service (PaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Run your applications reliably and resilient.</li>
<li>Orchestrates running your app over multiple nodes
<ul>
<li>Keeps it available</li>
<li>Keeps it performant by scaling</li>
<li>Performs rolling upgrades</li>
</ul>
</li>
<li>Can run
<ul>
<li>Any executable (type of apps)</li>
<li>Reliable services (that persist state)</li>
<li>Actor-based applications (that shares the state)</li>
<li>And orchestrate containers</li>
</ul>
</li>
<li>Runs in the <code class="language-plaintext highlighter-rouge">cloud</code> and <code class="language-plaintext highlighter-rouge">on-premises</code></li>
<li>In production you must need minimum 5 VMs to run it reliably.</li>
<li>Can run <code class="language-plaintext highlighter-rouge">serverless</code> as <strong>Service Fabric</strong> <strong>Mesh</strong> service. You can spin up Service Fabric without using 5 VMs.</li>
</ul>
<p class="notice--success">Did you know Azure Service Fabrics is a technology that Microsoft uses itself to run many of their own Azure Services, like <code class="language-plaintext highlighter-rouge">Azur e SQL Databases</code>? Learn more about <a href="https://azure.microsoft.com/en-us/services/service-fabric/">Azure Service Fabrics</a></p>
<h3 id="cloud-services">Cloud Services</h3>
<p><img src="https://imgur.com/6cIWhPK.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Cloud Services</code> is under <strong>Platform as a Service (PaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Run your applications in VMs and manages in the form of Web worker or Worker roles.</li>
<li>Abstracts the VM so that you don’t have to deal with OS or network.
<ul>
<li>It provides scaling</li>
</ul>
</li>
<li>You package your app and deploy it using VS template
<ul>
<li>To run as an HTTP-based application as web worker role, or</li>
<li>To run as a background job as worker role.</li>
</ul>
</li>
<li>Enable you to have some control over the VM
<ul>
<li>RDP into the VMs you can create task to install things in VMs.</li>
</ul>
</li>
</ul>
<p class="notice--success">Did you know <strong>Azure Cloud Services</strong> was one of the first <code class="language-plaintext highlighter-rouge">Platform-as-a-Service</code> offering from Azure and takes care the management of VMs for you? Learn more about <a href="https://azure.microsoft.com/en-us/services/cloud-services/">Azure Cloud Services</a></p>
<h3 id="azure-app-services-web-apps">Azure App Services Web Apps</h3>
<p><img src="https://imgur.com/rsLK6zD.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure App Services Web Apps</code> is under <strong>Platform as a Service (PaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Runs your app in a Webserver-as-a-service like <code class="language-plaintext highlighter-rouge">IIS</code> or <code class="language-plaintext highlighter-rouge">Tomcat</code></li>
<li>.Net, Java, PHP, Node.js and Python</li>
<li>App Service features
<ul>
<li>Continuous Deployment</li>
<li>Custom Domains</li>
<li>Deployment Slots no downtime</li>
<li>(automatic) scaling</li>
<li>Authentication/ Authorization</li>
<li>WebJobs</li>
<li>Hybrid connections to on-premise resources ( like connect your own data-center database)</li>
</ul>
</li>
<li>SLA 99.95% even you run only one instance.</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/app-service/web/">Azure Web Apps</a></p>
<h3 id="azure-app-services-webjobs">Azure App Services WebJobs</h3>
<p><img src="https://imgur.com/gGrP5wI.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure App Services WebJobs</code> is under <strong>Platform as a Service (PaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Runs your background task in App Services</li>
<li>Runs inside your App Service
<ul>
<li>Consumes resources</li>
<li>Is dependent on your App Services ( If WebJob as a part of a Web App the WebJob will stop if Web App is stopped )</li>
</ul>
</li>
<li>Can be triggered
<ul>
<li>Continuously</li>
<li>On a schedule ( like every 10 mins)</li>
<li>By outsider resources
<ul>
<li>New message on a queue (WebJob SDK has in-build code to connect to Queues)</li>
</ul>
</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://docs.microsoft.com/en-us/azure/app-service/webjobs-create">Azure App Services WebJob</a></p>
<h3 id="azure-app-services-mobile-apps">Azure App Services Mobile Apps</h3>
<p><img src="https://imgur.com/ntXbHFU.png" alt="" class="full" /></p>
<p class="notice--info notice--success">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure App Services Mobile Apps</code> is under <strong>Platform as a Service (PaaS)</strong> offering from Microsoft Azure.</p>
<ul>
<li>Run your app as a backend for mobile clients</li>
<li>You can create backend app using <code class="language-plaintext highlighter-rouge">.Net or Node.js</code></li>
<li>Mobile clients connect with the SDK
<ul>
<li>iOS, Android, Windows, Xamarin Forms</li>
</ul>
</li>
<li>Unique features offered by Mobile Apps that is not in Web Apps
<ul>
<li><strong>Offline sync</strong> which enables the mobile app to loose connection to backend still continue working and sync back when connected to backend eventually )</li>
<li><strong>Push notifications</strong> which enables you to send notifications to your mobile app like you get when you receive new email.</li>
</ul>
</li>
<li>All of the App Service features
<ul>
<li>Deployment Slots</li>
<li>(Automatic) scaling</li>
</ul>
</li>
<li>SLA 99.95%</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/app-service/mobile/">Azure App Services Mobile Apps</a></p>
<h3 id="function-apps">Function Apps</h3>
<p><img src="https://imgur.com/Q7lOW1P.png" alt="" class="full" /></p>
<p class="notice--info notice--success">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Function Apps</code> is under <strong>Serverless or Logic as a Service</strong> offering from Microsoft Azure.</p>
<ul>
<li>This is also part of <code class="language-plaintext highlighter-rouge">Azure App Services</code></li>
<li>Executes small pieces of code like execute whenever file get uploaded.</li>
<li>Can be triggered
<ul>
<li>Continuously</li>
<li>On a schedule ( like every 10 mins)</li>
<li>By outsider resources
<ul>
<li>Web Hooks</li>
<li>New messages in a queue</li>
</ul>
</li>
</ul>
</li>
<li>Have bindings
<ul>
<li><strong>Incoming</strong> Azure Blob value</li>
<li><strong>Outgoing</strong> Azure Storage Queue Message</li>
</ul>
</li>
<li>All of the App Service features
<ul>
<li>Deployment Slots</li>
<li>Continuous Deployment</li>
<li>(Automatic) scaling</li>
</ul>
</li>
<li>Can run in consumption mode (serverless)
<ul>
<li>Pay only for their execution</li>
<li>Scale automatically</li>
</ul>
</li>
</ul>
<p class="notice--success">In consumption mode Azure Functions become ideal after 5 mins of non-running. Learn more about <a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-overview">Azure Functions</a></p>
<h3 id="azure-logic-apps">Azure Logic Apps</h3>
<p><img src="https://imgur.com/6ocXvSn.png" alt="" class="full" /></p>
<p class="notice--info">💁 <strong>Important</strong> <br />
<br />
<code class="language-plaintext highlighter-rouge">Azure Logic Apps</code> is under <strong>Serverless or Logic as a Service</strong> offering from Microsoft Azure.</p>
<ul>
<li>Execute a pipeline of tasks in a process</li>
<li>Replaces for BizTalk Services</li>
<li>Can be triggered
<ul>
<li>Continuously</li>
<li>On a schedule ( like every 10 mins)</li>
<li>By outsider resources
<ul>
<li>Web Hooks</li>
<li>New messages in a queue</li>
</ul>
</li>
</ul>
</li>
<li>After being triggered Azure Logic app goes on to execute a process by calling connectors which are basically APIs.
<ul>
<li>Many connectors to resources
<ul>
<li>Like connectors to Office 365, Twitter, SendGrid, or your own APIs.</li>
</ul>
</li>
</ul>
</li>
<li>Only run in <strong>consumption mode</strong> (serverless)
<ul>
<li>Scales automatically</li>
</ul>
</li>
</ul>
<p class="notice--success">Learn more about <a href="https://azure.microsoft.com/en-us/services/logic-apps/">Azure Logic Apps</a></p>
<h2 id="when-to-use-what">When to use What?</h2>
<h3 id="based-on-high-level-application-requirement">Based on high level application requirement</h3>
<table>
<thead>
<tr>
<th>When</th>
<th>Use these Azure Service</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Full control</strong> & responsibility, Stay vendor agnostic (run anywhere vendor agnostic)</td>
<td><strong>VMs, Container Instances, Kubernetes Service, Web App for Containers, Azure Batch, Service Fabric (Mesh)</strong></td>
</tr>
<tr>
<td><strong>Azure manages</strong> your app, You manage scaling and configuration</td>
<td><strong>Batch, Cloud Services, Mobile App, Web App, Service Fabric</strong></td>
</tr>
<tr>
<td>Be able to <strong>RDP</strong> in VMs</td>
<td><strong>Batch, Cloud Services, Service Fabric(Mesh)</strong></td>
</tr>
<tr>
<td><strong>Azure manage</strong> your app, <strong>including</strong> <strong>scaling</strong>, You manage configuration</td>
<td><strong>Azure Function App, Logic App</strong></td>
</tr>
</tbody>
</table>
<h3 id="comparing-container-options-and-vms">Comparing Container Options and VMs</h3>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/YPcnULJ.png" alt="" /> VM</th>
<th><img src="https://imgur.com/5IrDqJM.png" alt="" /> Container Instances</th>
<th><img src="https://imgur.com/ukowMIs.png" alt="" /> Kubernetes Services</th>
<th><img src="https://imgur.com/qwLOnPT.png" alt="" /> WebApp for Containers</th>
<th><img src="https://imgur.com/3HK1rtT.png" alt="" /> Batch</th>
<th><img src="https://imgur.com/cdq4W3I.png" alt="" /> Service Fabric</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Lift-and shift your app to the cloud</strong></td>
<td>✔️</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Simple apps (without orchestrator)</strong></td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Complex apps (with orchestrator)</strong></td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Run orchestrated microservices</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Run anywhere</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Run web app on Linux with App Service features</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Repetitive jobs (on massive scale)</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<h3 id="where-to-run-your-background-tasks">Where to Run your Background Tasks?</h3>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/eIRJkJj.png" alt="" /> Cloud Services</th>
<th><img src="https://imgur.com/OcHd8Qi.png" alt="" /> Web Jobs</th>
<th><img src="https://imgur.com/ZWWXH9G.png" alt="" /> Functions</th>
<th><img src="https://imgur.com/0nJeR1T.png" alt="" />Batch</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Short-running tasks</strong></td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Long-running tasks</strong></td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Resource intensive tasks</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Pay all month</strong></td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Pay only for what and when you run</strong></td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>Need to deploy a complete app to run</strong></td>
<td>✔️</td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Can deploy only the code necessary to run</strong></td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<h3 id="where-to-run-your-applications">Where to Run your Applications?</h3>
<table>
<thead>
<tr>
<th>Scenarios</th>
<th><img src="https://imgur.com/iH0hVvs.png" alt="" /> Web App</th>
<th><img src="https://imgur.com/SRk7YqX.png" alt="" /> Mobile App</th>
<th><img src="https://imgur.com/eIRJkJj.png" alt="" />Cloud Services</th>
<th><img src="https://imgur.com/cdq4W3I.png" alt="" /> Service Fabric</th>
<th><img src="https://imgur.com/j1Y4A4B.png" alt="" />Functions</th>
<th><img src="https://imgur.com/z0fI8LX.png" alt="" /> Logic App</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Host web applications</strong></td>
<td>✔️</td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Host your own APIs</strong></td>
<td>✔️</td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>Host backend for mobile apps</strong> (push notification, offline sync)</td>
<td> </td>
<td>✔️</td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr>
<td><strong>Automate one step of a process</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>Automate a complete process</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
</tr>
<tr>
<td><strong>Feature deployment slots, Auth</strong></td>
<td>✔️</td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
<td>✔️</td>
<td> </td>
</tr>
<tr>
<td><strong>Run (micro) services at massive scale</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td>✔️</td>
<td>✔️</td>
<td> </td>
</tr>
</tbody>
</table>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariLearn what service from Azure you should use while creating your application in Azure. This article has quick short answer for when to use what in Azure service? Get free e-book on Design Distributed Systems. It is difficult to compare options available in Azure. The comparison with the features of the options provides you tools to choose the right service with your requirements.Virtualization Basics with VMware2021-11-24T00:00:00+00:002021-11-24T00:00:00+00:00https://www.rupeshtiwari.com/introduction-to-virtualization<blockquote>
<p>Virtualization is the single concept that now-a-days cloud service providers are using at massive scale and delivering IT technologies such as storage, compute, networking, database, analysis and much more as a service via internet and on-demand basis as pay-as-you-go pricing model. Therefore, it is very important that you must understand what is virtualization.</p>
</blockquote>
<h2 id="what-is-virtualization">What is Virtualization</h2>
<p>Virtualization is a technology that lets you create a logical boundary around physical hardware and capabilities such as compute, storage and network.</p>
<p>Suppose you got one bare machine that has 64 GB RAM, 2 TB storage and 32 Core CPU. Then using hypervisor or virtualization techniques you can create 10 virtual machines on this hypervisor each having 4 GB RAM, 100GB disk storage and 2 Core CPU. Each Virtual Machine is a literal isolated machine, they can not talk to each other unless you provision so it gives data center level infrastructure.</p>
<p><img src="https://i.imgur.com/yClGm4N.png" alt="" /></p>
<h2 id="what-is-a-host">What is a Host?</h2>
<p>Host is a hardware where a hypervisor is installed. Guest is a virtual machine created on a hypervisor. Virtual Machine operating systems are also called Guest Operating Systems.</p>
<h2 id="what-is-a-hypervisor">What is a Hypervisor?</h2>
<p>Hypervisor is a software or firmware that creates and runs virtual machines. A hypervisor is also called as Virtual Machine Manager (VMM)</p>
<p><img src="https://i.imgur.com/fSWfhNW.png" alt="" /></p>
<p>There are 2 types of hypervisor:</p>
<ol>
<li>Type-1</li>
<li>Type-2</li>
</ol>
<p><img src="https://i.imgur.com/581Jh8v.png" alt="" /></p>
<h3 id="what-are-the-attributes-of-type-1-hypervisor">What are the attributes of Type-1 Hypervisor? </h3>
<ul>
<li>They come with bare metal machine on top of we install hypervisor firmware</li>
<li>Type-1 Hypervisor is used for enterprise data center</li>
<li>VMWare has ESXi software for hypervisor</li>
<li>Microsoft has Hyper-V</li>
<li>Citrix has Xenserver</li>
</ul>
<p><img src="https://i.imgur.com/NugWBgI.png" alt="" /></p>
<h3 id="what-are-the-attributes-of-type-2-hypervisor">What are the attributes of Type-2 Hypervisor? </h3>
<ul>
<li>These types of hypervisors are softwares that you install on top of the Operating System of a computer. They are hosted hypervisors.</li>
<li>Type-2 Hypervisor is used for Testing, Learning purpose</li>
<li>VMWare has Workstation software for hypervisor</li>
<li>Microsoft has Virtual PC</li>
<li>Oracle has Virtual Box</li>
</ul>
<p><img src="https://i.imgur.com/kUsDd8n.png" alt="" /></p>
<h2 id="what-is-vmware-vsphere">What is VMWare VSphere?</h2>
<p>VMWare is an American cloud computing & virtualization company. Vsphere is the brand name of vmware suite of virtualization products.</p>
<p>VSphere has got:</p>
<ol>
<li>Vmware ESXi (Hypervisor)</li>
<li>Vcenter Server</li>
<li>Vsphere client</li>
<li>Vsphere web client</li>
</ol>
<h3 id="vsphere-client">Vsphere Client</h3>
<p>VSphere client is an User Interface to access & configure ESXi Host.</p>
<p><img src="https://i.imgur.com/lGU4gRX.png" alt="" /></p>
<h3 id="vcenter-server">Vcenter Server</h3>
<p>VCenter Server provides features like:</p>
<ul>
<li>Distributed Resource scheduler (DRS)</li>
<li>High Availability (HA)</li>
<li>VMotion ( replication technique )</li>
<li>Fault Tolerance</li>
<li>VM Migrations</li>
</ul>
<p><img src="https://i.imgur.com/3pQBmuQ.png" alt="" /></p>
<h2 id="what-is-a-data-store-in-vmware-hypervisor">What is a data store in VMWare Hypervisor? </h2>
<p>All files related to a virtual machine like log file, virtual disk file, iso files, configuration files are stored in the Data Store only. It can have 2 types of filesystems like Virtual Machine File System (VMFS) and NFS (Network File System).</p>
<p><img src="https://i.imgur.com/w0lx1WD.png" alt="" /></p>
<h2 id="what-is-migration-and-their-types">What is Migration and their types?</h2>
<p>Migration is the technique of moving a virtual machine from one host to another host or from one data store to another data store.</p>
<p>There are <code class="language-plaintext highlighter-rouge">5 Migration</code> strategies:</p>
<ol>
<li>Cold Migration</li>
<li>Suspended Migration</li>
<li>VMotion</li>
<li>P2V (Physical to Virtual)</li>
<li>V2V (Virtual to Virtual)</li>
<li>From Microsoft hyper-v to vmware</li>
<li>From vmware to xenserver etc.</li>
</ol>
<h2 id="what-are-the-states-of-a-virtual-machine">What are the states of a virtual machine?</h2>
<ol>
<li><strong>Hot</strong> State means virtual machine is <strong>ON</strong></li>
<li><strong>Cold</strong> State means virtual machine is <strong>OFF</strong></li>
<li><strong>Suspended</strong> State means virtual machine is <strong>ON</strong> <em>and</em> <strong>PAUSED</strong></li>
</ol>
<h2 id="what-is-cold-migration-technique">What is Cold Migration Technique?</h2>
<p>In the cold migration technique we transfer a VM while it is Cold State to another host. If 2 data centers are in the same Vcenter server instance then only cold migration is possible.</p>
<h2 id="what-is-suspended-migration-technique">What is Suspended Migration Technique?</h2>
<p>When you migrate your virtual machine (VM) in suspended state. Here power is on and running so it is a HOT migration category. Suspended state means your vm is in paused state and you resume from the same point at a later time. This migration is used to troubleshoot your virtual machine technical issues.</p>
<h2 id="what-is-vmotion">What is vMotion?</h2>
<p>vMotion is a live migration technique where you migrate your virtual machine at power on state or Hot State.</p>
<p>2 types of vMotion</p>
<ol>
<li>Host level: Where VM is migrated to another ESXi host</li>
<li>Data Store: Where VM is migrated from one datastore to another datastore.</li>
</ol>
<h2 id="what-is-physical-to-virtual-p2v-migration">What is Physical to Virtual P2V Migration?</h2>
<p>P2V migration converts a physical computer to a virtual machine. You run vmware vcenter converter and copy the physical server to VM in ESXi host.</p>
<p>In Microsoft Azure you use Azure Migration Service and agents to migrate your physical machine to the Azure cloud.</p>
<p>In AWS you also use a migration service to migrate your physical machine to AWS cloud.</p>
<h2 id="what-is-physical-to-virtual-v2v-migration">What is Physical to Virtual V2V Migration?</h2>
<p>V2V migration is like P2V except the source machine is already a virtual machine. Different company Virtual machine migrated to another new company. Like Hyper-V migrated to ESXi and Xenserver migrated to Hyper-V.</p>
<p>Microsoft Azure gives this facility to migrate Hyper-V or ESXi migration to the cloud. Most of the cloud service providers are giving this opportunity nowadays.</p>
<h2 id="what-is-fault-tolerance-ft-how-ft-works-what-are-ft-benefits">What is Fault Tolerance (FT), How FT works, What are FT benefits?</h2>
<p>Fault Tolerance is a Virtual Machine level concept where you protect your Virtual Machine from stop working. Suppose some operating system level corruption happened and your virtual machine is not working so you want to protect your VM from that and you create a redundant VM to make it more fault tolerant.</p>
<p>In VMWare you can enable/disable FT at VM level. FT makes system high availability give 0% downtime since VM will always be in running mode & no restart required in case of VM crash. Because you keep the VM in backup as a redundant VM. so billing will be for 2 VMs this is a costly solution.</p>
<p>FT must be used for Mission critical applications and servers like</p>
<ol>
<li>Auto Pilot system</li>
<li>Spacecraft mission</li>
</ol>
<p>The secondary VM is created by DRS ( Distributed Resource Scheduler ) in a different HOST than the primary VM host. VMware <a href="https://en.wikipedia.org/wiki/Lockstep_(computing)">lockstep technolog</a>y is used in FT where current state & events on primary VM is replicated on secondary VM. When primary goes down the secondary VM takes over and continues operating.</p>
<p><img src="https://i.imgur.com/6j97d3H.png" alt="" /></p>
<p><img src="https://i.imgur.com/xDIjbIO.png" alt="" /></p>
<p>FT avoids <a href="https://en.wikipedia.org/wiki/Split-brain">Split Brain</a> situation when primary fails then if 2 secondary VMs created and works is a problem called split brain. Primary and secondary continuously check each other’s heart-beat.</p>
<p>In Azure cloud you get Fault Domain where if you have more than 1 Virtual Machine then Azure will put your VM in 2 different fault domains therefore, if one VM is out of order then another VM will automatically start responding. This gives you any kind of hardware protection such as network or power on rack etc.</p>
<p>In Microsoft Azure cloud you also have a virtual machine scale set that gives you managed service internal it composed of load balancer, and more than 1 virtual machine. You can put them in different fault domains and update domains.</p>
<p>In Amazon AWS you get an availability group that is this same as the Azure availability set.</p>
<h2 id="what-is-high-availability">What is High Availability?</h2>
<p>You want your Virtual Machine to keep running regardless of the situation occurring in your data center. If you bought a 3rd party service like fastcomet or any other provider then it’s their responsibility to take care of High Availability (HA).</p>
<p>Therefore companies create Service Level Agreement (SLA) with customers to provide guaranteed service.</p>
<table>
<thead>
<tr>
<th>Level of Agreement</th>
<th>Down Time Per Hours</th>
</tr>
</thead>
<tbody>
<tr>
<td>99%</td>
<td>87 hours</td>
</tr>
<tr>
<td>99.9%</td>
<td>8.76 hours</td>
</tr>
<tr>
<td>99.99%</td>
<td>52 mins</td>
</tr>
<tr>
<td>99.999%</td>
<td>5 mins</td>
</tr>
</tbody>
</table>
<p>If SLA is not achieved then the company will give you compensation as per the level of agreement. Like Microsoft Azure will credit the money back to your monthly bill if they are not able to meet SLA. Similarly Amazon AWS also will compensate if they could not meet committed SLA.</p>
<h2 id="how-high-availability-ha-works-in-vmware">How High Availability (HA) works in VMWare? </h2>
<p>If your hardware or ESXi host has any issues then all VMs will become down. It is just like your entire data center is down. HA works on HOST level and Fault Tolerance (FT) works on VM level.</p>
<p>If you create a backup virtual machine then it is a very costly solution. In the past VM backup used to be the only solution therefore, companies used to avoid going on virtualization.</p>
<p>However, in VMWare High Availability meaning, if the host crashes or fails then all the VMs on that host get restarted from another new host and hence you get business continuity and high availability.</p>
<p>HA is an automated process that does not need any admin/human interference. In HA you do not need any standby ESXi hosts or extra VMs as redundant. It can restart VM on any runnin ghosts. HA doesn ot use VMotion and can be enabled on cluster settings.</p>
<p><img src="https://i.imgur.com/HOFLW0n.png" alt="" /></p>
<p>For HA you need below things in your VMWare</p>
<ol>
<li>Cluster</li>
<li>Shared Storage</li>
<li>VCenter Server configured for the environment.</li>
</ol>
<h3 id="prerequisites-for-vmware-vsphere-ha">Prerequisites for VMWare Vsphere HA</h3>
<p>All hosts must be licensed and the same version. You need a minimum of 2 hosts. All hosts must have a static IP address after restart of VM IP should persist. VM must be located in shared data storage and they should not have their own local data storage. All hosts in VMware HA Cluster must have DNS configured. HA works on <a href="https://en.wikipedia.org/wiki/Primary/replica_architecture">Master and Slave Architecture.</a></p>
<p><img src="https://i.imgur.com/zKqBVY0.png" alt="" /></p>
<h3 id="what-is-vmware-high-availability-ha-cluster">What is VMWare High Availability (HA) Cluster?</h3>
<p>When HA is enabled on the cluster then the election process occurs among all hosts. One host that has a maximum number of data stores mounted becomes master & rest become slaves.If Master host is down then again election will start.</p>
<p><img src="https://i.imgur.com/luhBFUQ.png" alt="" /></p>
<h2 id="what-is-a-resource-check-in-vmware-ha">What is a Resource check in VMware HA? </h2>
<p>Resource check is a feature of HA which ensures that capacity is available to restart VMs in a crash scenario. Also it reserves some capacity that you can not use when HA is enabled. Suppose you have 64 GB ram and you have 5 VMs each 4 GB ram then it may reserve extra 20 GB ram for crash scenarios.</p>
<h2 id="what-is-ha-failover-time">What is HA Failover Time?</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
HA Failover time = Vsphere web client started responding time - Vcenter server VM stopped responding time.
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">64</code> hosts and <code class="language-plaintext highlighter-rouge">6000</code> VMs will take approximately <code class="language-plaintext highlighter-rouge">7 minutes</code> to failover.</p>
<p><img src="https://i.imgur.com/9HFdcT3.png" alt="" /></p>
<h2 id="what-is-virtual-machine-template-in-vmware">What is Virtual Machine Template in VMWare?</h2>
<p>Convert fully configured custom personalized VM into VM Template. Use a VM template to deploy a large number of the same VMS.</p>
<p>Nowadays every cloud service provider including (Azure, AWS) supports VM templates in various forms.</p>
<h2 id="summary">Summary</h2>
<p>Make sure you learn more about all the points that I mentioned in this article. I may be incorrect so feel free to suggest your opinions. Share some of your experience if you have used VMware or Hypervisor-V etc in your day-to-day work.</p>
<h2 id="reference">Reference</h2>
<ul>
<li><a href="https://www.youtube.com/watch?v=-VxzIIsFXFY&list=PLBGx66SQNZ8aiuWzEFavDMpQ1RRWsZLZV&ab_channel=TechnicalGuftgu">VMWare Virtualization Tutorial</a></li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariVirtualization is the single concept that now-a-days cloud service providers are using at massive scale and delivering IT technologies such as storage, compute, networking, database, analysis and much more as a service via internet and on-demand basis as pay-as-you-go pricing model. Therefore, it is very important that you must understand what is virtualization.What is Hyper-Visor?2021-11-24T00:00:00+00:002021-11-24T00:00:00+00:00https://www.rupeshtiwari.com/what-is-hyper-vRupesh TiwariCloud Security Best Practices for application, servers and network2021-11-22T00:00:00+00:002021-11-22T00:00:00+00:00https://www.rupeshtiwari.com/cloud-security-best-practices<blockquote>
<p>According to <a href="https://www.statista.com/statistics/1062879/worldwide-cloud-storage-of-corporate-data/#:~:text=As%20of%202021%2C%20around%2050,next%20to%20advancing%20business%20agility">Statista.com</a> As of 2021, around 50 percent of all corporate data is stored in the cloud. This share reached 30 percent in 2015 and has continued to grow as companies increasingly shift their resources into cloud environments in the hope of improving security and reliability next to advancing business agility. Therefore, securing your data center, your deployed workloads and your architecture in the cloud is now-a-days top priority for every company.</p>
</blockquote>
<p>Most of the Cloud providers like Amazon AWS & Microsoft Azure follow the shared security model where they take care of security of the cloud and you secure what is in the cloud. Therefore, you must educate yourself to start securing your workload and architecture in the cloud.</p>
<p>You start securing your environment from network to server to application & data layers. Let’s discuss all of these best practices in detail in this article. For cloud protection, I will refer to Azure cloud in this article. You can correlate <a href="https://docs.microsoft.com/en-us/azure/architecture/aws-professional/services">Azure Services & Infrastructures with AWS cloud</a> by referring to this article.</p>
<h2 id="application-security">Application Security</h2>
<p>Why is application security important in your on-premise or cloud? Did you know 75% of organizations worldwide have experienced some kind of phishing attack in 2021? Have you heard about a multi-layered approach to security? Cybercriminals know that web applications are the key to enter an organization’s technology enterprise and steal valuable information.</p>
<p>You can secure your web applications in the following ways:</p>
<ul>
<li>Single Sign-On</li>
<li>Application Integrity</li>
<li>Vulnerability Scanning and</li>
<li>Virtual Patching</li>
</ul>
<h3 id="single-sign-on-in-the-cloud">Single Sign-On in the cloud</h3>
<p>You have your on-premise or cloud line of business (LOB) applications, and you want to secure them. Single Sign-On (SSO) is one of the crucial steps to securing your applications. You may ask what Single Sign-On will do in terms of security, right? I used to think SSO is something where you enter a password or login to your enterprise network once, and then you don’t need to log in for other applications. Then what exactly is it saving for security? Well, it can protect you from phishing attacks, for one. According to <a href="https://www.google.com/url?q=https://www.tessian.com/blog/phishing-statistics-2020/&sa=D&source=editors&ust=1633966929713000&usg=AOvVaw0rPv3CiTK4qpNOEfoxv_3d">Tessian Phishing Statistics 2020</a>, last year, 75% of organizations worldwide faced a phishing attack. Azure integrates your web applications with Azure active directory and provides single sign-on.</p>
<p><img src="https://i.imgur.com/UOsaf4h.png" alt="" /></p>
<p>Phishing attacks are emails that will ask you to log in to your application using your password. Examples of phishing emails include: Please Read subject or Payment is Urgent Credential needed for login to secure etc. Suppose you were using SSO and strong authentication in your organization, eliminating the need for employees to ever manually enter passwords to access systems, applications, or information. An email requesting credentials would stand out as a likely phishing attack.</p>
<h3 id="application-integrity-check">Application Integrity Check</h3>
<p>In 2020, the number of data breaches in the United <a href="https://www.statista.com/statistics/273550/data-breaches-recorded-in-the-united-states-by-number-of-breaches-and-records-exposed/">States came in at a total of 1001 cases.</a></p>
<p>Therefore, as an organization, you want to make sure your application data integrity is conserved. Application Integrity helps clients secure their organizations. There are many best practices and rules available in Azure policy that require compliance from your applications. Basic requirements like using Https for web APIs and certification validation for calling endpoints are very effective. Limiting access to your applications for only those clients that have a valid certificate will limit your exposure. By default, incoming client certificates are disabled for Azure App Service web applications.</p>
<p>Azure has managed initiatives (collection of policies) for compliance domain and security control. Make sure you apply one of them to your app service plan and other resources in your workload.</p>
<h3 id="vulnerability-scans-in-applications">Vulnerability Scans in Applications</h3>
<p><a href="https://www.google.com/url?q=https://info.edgescan.com/vulnerability-stats-report-2021&sa=D&source=editors&ust=1633966929715000&usg=AOvVaw1-PJ5lIm1vsmfu9HDYW-8j">Edgescan’s 2021 Vulnerability Statistics Report</a> analyzed the severity of web application vulnerabilities. It found that 50 percent of internal application vulnerabilities are considered a high or critical risk. It also found that 32 percent of vulnerabilities in internet-facing applications are considered a high or critical risk. According to <a href="https://www.k2io.com/2021-verizon-data-breach-investigations-report-is-out/">the Verizon Data Breach Investigation Report</a> web applications remain the top vector used by hacking in breaches at over 90%.</p>
<p><img src="https://i.imgur.com/WbwSU43.png" alt="" /></p>
<p>Therefore, you must scan your web app for vulnerability risk. Azure Web apps provide built-in tools for diagnoses and solving vulnerability problems. Azure App Service Diagnostics will help identify and alert you to the security risks associated with your web application.</p>
<p><img src="https://i.imgur.com/lqGz7hu.png" alt="" /></p>
<h3 id="virtual-patching-for-legacy-code">Virtual Patching for Legacy Code</h3>
<p>Virtual Patches are meant for protecting unitary vulnerabilities that the current WAF Security Policy does not already protect.</p>
<p>Cybercriminals know web apps connect with backend, active directory to get valuable business and intellectual data. <a href="https://www.ibm.com/security/data-breach">According to IBM data breach report 2021</a>, data breach costs rose from USD 3.86 million to USD 4.24 million, a 17-year historic increase.</p>
<p>Due to the large volume of cloud migration and web application development, many organizations create too many web apps with too many vulnerabilities. They are left exposed to potential data breaches through these applications. Some companies even take their windows applications and convert them into web applications without understanding how to secure them. Additionally, many applications are poorly written and have security loopholes. So how does an organization mitigate the risk of a breach through their web app portfolio?</p>
<p>Use Microsoft’s PaaS (platform service) and leverage Azure vulnerability scans and patching services. Frequent scanning and patching ensure your protection is up-to-date and continuous. Need secure zero-day vulnerability on your web app? Review your case with Azure and get protection for your dedicated host.</p>
<p>In summary, I recommend you follow a <a href="https://aws.amazon.com/architecture/well-architected/?wa-lens-whitepapers.sort-by=item.additionalFields.sortDate&wa-lens-whitepapers.sort-order=desc">Well-Architected Framework</a> for security for web applications. Do code reviews and seek out any security violations in your code and remove them - similar to sanitizing your incoming text from client to server to make sure you avoid running scripts injected by malicious users.</p>
<p>What about vulnerabilities in existing and potentially out-of-date applications? What if you have VMs hosting web apps in the cloud and you own the responsibility of patching servers? If you leave your VMs vulnerable, they will be the first choice for attackers. The answer is Virtual Patching. Virtual Patching is the technique leveraging WAS (web application scan) and WAF (Web application firewall) to install virtual patch software; use WAS to identify vulnerabilities, and then automatically create rules in <a href="https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview">WAF (Web Application Firewall)</a>. This way, you end up protecting your app from existing vulnerabilities attacks without changing your source code.</p>
<h3 id="network-security-in-cloud">Network Security in Cloud</h3>
<p>In cloud you can secure your network by doing the following things:</p>
<ul>
<li>Network segmentation</li>
<li>Network Session Protection</li>
<li>Network Monitoring</li>
<li>Traffic Encryption</li>
</ul>
<p><strong>Network Segmentation</strong> inspired by <a href="https://en.wikipedia.org/wiki/Zero_trust_security_model">Zero Trust Model</a>.</p>
<p>Network Segmentation can help control your company’s network traffic flow. Your IT team can regulate who has access to which segment of the network. Segmentation improves security and performance by dividing the entire network into parts. For example, as a bank, you want to restrict branch employees from your financial reporting systems. With Network segmentation, you can limit traffic flow in the financial system segment, improving performance for the financial analysts and limiting unnecessary system access from unauthorized systems and users. Additionally, you may limit the scope of an infection or attack to a single segment rather than the entire network. Finally, Network Segmentation is a crucial element of the <a href="https://en.wikipedia.org/wiki/Zero_trust_security_model">Zero Trust model</a>.</p>
<p>Please see the <a href="https://securityscorecard.com/blog/network-segmentation-best-practices-to-maximize-cybersecurity">best practices of segmentation.</a></p>
<p>On-premises you can use a legacy approach to create multiple DMZ (demilitarized zones) using internal firewalls and Access control Lists( ACLs); however, this approach is more costly and time-consuming. Nowadays, you can apply tags on selected routes and group them virtually by tagging. A tag will enforce segmentation policy directly on the network equipment.</p>
<p>In Azure cloud, you can achieve network segmentation as well by properly organizing network infrastructures. You have an azure subscription, virtual network, network security group, application security group, and an azure firewall. These are great tools to create micro perimeters or segments.</p>
<p>In Azure cloud, you can use subnets within a single virtual network and apply custom routes on each subnet to restrict traffic flow. Alternatively, you can use an application gateway to each subnet. However, this pattern where all workload is in one virtual network cannot span multiple regions since the scope of a virtual network is limited to only one region.</p>
<p>When you use a virtual network (VNet), you get built-in segmentation because one VNet or VPC by default can not talk to others unless you set up peering. You can set up rules like, for example, virtual network X can’t talk with virtual network Y but can talk with virtual network Z, or no Internet for Virtual network X except for access to <em>.github.com</em>, and so on.</p>
<p>You can use multiple Virtual networks and use virtual network peering to get segmentation free and use NSG or ASG to enforce policies. However, Virtual network peering is not transitive by nature so consider <strong>transitive gateway</strong> or <strong>Hub & Spoke</strong> model.</p>
<p><img src="https://i.imgur.com/dPSq0qG.png" alt="" /></p>
<p>To fix transitive issues, I would recommend going to a Hub and Spoke topology where you create one dedicated VNet as your hub network and all traffic passes through the virtual hub network, and it can act as a gateway to other hubs in different regions. You can set up your security posture at the hubs, so they get to segment and govern the traffic between the virtual networks in a scalable way. Adding a new workload or new virtual network with the same security posture is a minimal effort. <a href="https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/hybrid-networking/network-level-segmentation">Learn more about network segmentation in azure here.</a></p>
<h3 id="session-protection">Session Protection</h3>
<p>Your users might log in to the system to establish a session with the server and forget to close the browser, creating <a href="https://www.google.com/url?q=https://www.globalsign.com/en/blog/session-hijacking-and-how-to-prevent-it&sa=D&source=editors&ust=1633966929727000&usg=AOvVaw2rIMES5rLflGwVo0CM3CFM">Session Hijacking</a> opportunities. Any cybercriminal can take advantage of the open sessions and steal valuable information. According to the 2021 <a href="https://www.globalsign.com/en/blog/session-hijacking-and-how-to-prevent-it">Vulnerability Statistics Report</a>, 37% of XSS attacks targeted web applications. Those attacks could be prevented by Network session protection.</p>
<p><img src="https://i.imgur.com/W7Ifi3s.png" alt="" /></p>
<h4 id="types-of-session-hijacking-attacks">Types of Session Hijacking Attacks:</h4>
<ul>
<li>Cross-Site Scripting (XSS): attackers exploit vulnerabilities within servers and inject scripts (JS, Active Directory, DOS) via web pages and retrieve information.</li>
<li>Session-Side Jacking: attackers can sniff through network packets to get the session key via session cookies and impersonate them to perform malicious actions. These attacks are even more likely when employees access company assets via public WIFI or an unsecured hotspot.</li>
<li>Session Fixation: Attackers supply their session key and spoof the user into accessing a vulnerable server.</li>
</ul>
<p>You can protect your network by using a web application firewall to protect any session established on your network from the outside world. Also, if you apply initiatives and policies to your resource groups, you will be forced to implement certification-based communication. That way, you verify the outside endpoints connecting to your server and mitigate session attacks. <a href="https://docs.microsoft.com/en-us/azure/app-service/configure-ssl-bindings#add-a-certificate-for-custom-domain">In the Azure web app, you can also apply for SSL certificates</a>.</p>
<p><img src="https://i.imgur.com/mt7RsG1.png" alt="" /></p>
<h3 id="network-monitoring">Network Monitoring</h3>
<p>Network security monitoring is helpful for detecting and analyzing potential threats. Network cybersecurity monitoring can help protect your enterprise data from malicious actors and hackers, from business stats to personal user information. In your on-premises solution, you can use the NMap tool to scan the network.</p>
<p><img src="https://i.imgur.com/hCZwCBu.png" alt="" /></p>
<p><a href="https://docs.microsoft.com/en-us/azure/azure-monitor/insights/network-insights-overview">Azure cloud Network Insights</a> within the Azure Security Center can comprehensively view health and metrics for all deployed network resources. You can also see dependency flow, connectivity, and traffic flow, and more.</p>
<h3 id="network-traffic-encryption">Network Traffic Encryption</h3>
<p>Encrypting your data while it’s in transit is an essential step toward securing your applications. You can purchase certificates from a certificate authority and use them to encrypt the messages that pass in and out of your servers. Encryption prevents unauthorized users from intercepting and examining the information in these messages while in transit. Encryption also prevents <a href="https://en.wikipedia.org/wiki/Man-in-the-middle_attack">Man in the Middle Attacks</a>.</p>
<p>Use Application Gateway or front door in Azure or the Web Application Firewall of AWS to protect your traffic and ensure they are encrypted. If you need end-to-end encryption, Application Gateway can use your private key to decrypt the traffic on the gateway and re-encrypt using the public key of the service running in the backend pool.</p>
<p>Exposing your website or web application through Application Gateway means you don’t directly connect your servers to the web. You’re exposing only port 80 or port 443 on the application gateway. Your web servers aren’t directly accessible from the internet, reducing the attack surface of your infrastructure.</p>
<p><img src="https://i.imgur.com/uTwY2Zx.png" alt="" /></p>
<p>Application Gateway can implement an SSL connection with clients. Application Gateway can also implement an SSL connection with the servers running your application.</p>
<h3 id="serverhost-security-in-cloud">Server/Host Security in cloud</h3>
<p>In the cloud, you have to secure your virtual machines on your own. Cloud providers will take care of the cloud, but you have to take care of whatever is in the cloud, especially virtual machines. When you provision virtual machines, you are responsible for patching and installing anti-virus and otherwise caring for security.</p>
<p>Azure Security center can scan the network and, based on security policy, alert you to any pending patches, including cloud and on-premise malware vulnerabilities found. Of course, you still have to take corrective actions.</p>
<p><strong>Please consider below best practices to secure your EC2 instances or Virtual Machines in cloud:</strong></p>
<ul>
<li>Installing Anti Malware & Antivirus, you can purchase them from Microsoft or Symantec, etc.</li>
<li>Secure your encryption keys created in VMs in the Azure Key Vault or AWS key management service (KMS) in AWS cloud.</li>
<li>Protect your server by running frequent backups - try <a href="https://docs.microsoft.com/en-us/azure/backup/backup-overview">Azure Backup</a> which does not need any CaPeX to set up and provides complete protection of your application data.</li>
<li>Protect from unplanned outages by implementing Site Recovery to ensure your organization’s business continuity and disaster recovery objectives keeping your applications and servers running. <a href="https://docs.microsoft.com/en-us/azure/site-recovery/site-recovery-overview">Azure Site Recovery</a> helps to protect both on-premise and cloud workloads from disasters.</li>
<li>
<p>Secure SQL data using Transparent Data Encryption (TDE) and column level encryption (CLE).</p>
</li>
<li>Encrypt Virtual Machine disks - to encrypt your VMs disks try <a href="https://docs.microsoft.com/en-us/azure/security/fundamentals/azure-disk-encryption-vms-vmss">Azure Disk Encryption</a> solution.</li>
<li>Consistently and frequently run Patch updates.</li>
<li>Prevent unauthorized access on your VMs by implementing SSO (Single Sign On) and using identity-based access controls.</li>
</ul>
<h2 id="summary">Summary</h2>
<p>Finally, in this article we learned that your cloud service provider will take care of the security of the cloud. However, You are responsible for securing workloads in the cloud.</p>
<p>In order to secure your workload in the cloud you must follow both Defense in depth and Zero Trust Model. The objective of Defense in Depth is to protect information and prevent it from being stolen by those who aren’t authorized to access it. Zero Trust is a strategic initiative that helps prevent successful data breaches by eliminating the concept of trust from an organization’s network architecture. You trust nobody in your network, create a micro perimeter. This will help to Bring Your Device (BYOD) related security risk.</p>
<h2 id="references">References</h2>
<ul>
<li>https://www.comparitech.com/blog/information-security/cybersecurity-vulnerability-statistics/</li>
<li>https://blog.qualys.com/product-tech/2017/05/04/virtual-patching-a-lifesaver-for-web-app-security</li>
<li>https://success.qualys.com/discussions/s/article/000006325</li>
<li>https://www.microsoft.com/security/blog/2019/10/23/- perimeter-based-network-defense-transform-zero-trust-model/</li>
<li>https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/hybrid-networking/- network-level-segmentation</li>
<li>https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview</li>
<li>https://docs.microsoft.com/en-us/azure/azure-monitor/insights/network-insights-overview</li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariAccording to Statista.com As of 2021, around 50 percent of all corporate data is stored in the cloud. This share reached 30 percent in 2015 and has continued to grow as companies increasingly shift their resources into cloud environments in the hope of improving security and reliability next to advancing business agility. Therefore, securing your data center, your deployed workloads and your architecture in the cloud is now-a-days top priority for every company.Azure Network Watcher Basics2021-11-20T00:00:00+00:002021-11-20T00:00:00+00:00https://www.rupeshtiwari.com/azure-network-watcher-basics<blockquote>
<p>Azure Network Watcher is a <strong>monitoring service</strong> with lots of other important services for network. <strong>Most network diagnostics issues can be detected and analyzed with Azure Network Watcher</strong>. Learn everything about Network Watcher in this article.</p>
</blockquote>
<h2 id="what-network-watcher-does">What Network Watcher Does?</h2>
<p><img src="https://imgur.com/H2mtWg4.png" alt="" class="align-center" /></p>
<ul>
<li>You use Network Watcher to monitor and repair the network health of IaaS (Infrastructure as Service) resources.</li>
<li>It can capture network packets by sniffing the network</li>
<li>It can analyze and display the network topology</li>
<li>It can audit the security rules of network and VMs.</li>
<li>It is a regional service</li>
<li>It is automatically enabled in a region when you create or update a virtual network (vNet) in it.</li>
</ul>
<h2 id="capabilities-of-network-watcher">Capabilities of Network Watcher</h2>
<h3 id="monitoring">Monitoring</h3>
<ul>
<li>Network Watcher can monitor communication between a VM and another endpoint (could be VM).</li>
<li>Network Watcher can view resources in a virtual network.</li>
</ul>
<h3 id="diagnostics">Diagnostics</h3>
<ul>
<li>It can diagnose network traffic filtering problems to or from a VM.</li>
<li>It can diagnose network routing problems from a VM.</li>
<li>It can diagnose outbound connections from a VM.</li>
<li>It can capture packets to and from VM.</li>
<li>It can determine relative latencies between Azure regions and Internet Service Providers (ISPs).</li>
<li>It can show security rules for an interface like network interface for a VM.</li>
</ul>
<h3 id="metrics-and-logs">Metrics and Logs</h3>
<ul>
<li>It can analyze network security groups traffic.</li>
<li>It can show diagnostics logs for network resources.</li>
</ul>
<h2 id="network-watcher-in-azure-portal">Network Watcher in Azure Portal</h2>
<h3 id="topology">Topology</h3>
<p><img src="https://imgur.com/aMuidSb.png" alt="" class="align-center" />
You can monitor the topology of your virtual network and you can click on any component to view them.</p>
<h3 id="connection-monitor">Connection Monitor</h3>
<p><img src="https://imgur.com/1XfzXrM.png" alt="" class="align-center" />
Connection Monitor monitors the health of the connections between 2 endpoints could be 2 virtual networks or 2 VMs.</p>
<h3 id="next-hop">Next hop</h3>
<p>This will give you the next network hop from a given source IP address to a destination IP address. You can use this to troubleshoot the issues.</p>
<p><img src="https://imgur.com/dvmPfCr.png" alt="" class="align-center" />
Example: Source IP address could be your VM in azure and destination IP address could be your local home machine IP address.</p>
<p><img src="https://imgur.com/eYveirr.png" alt="" class="align-left" />
So in this example the next hop is Internet.</p>
<h3 id="effective-security-rules">Effective Security Rules</h3>
<p><img src="https://imgur.com/vr6xfiR.gif" alt="" class="align-center" /></p>
<p>Check security settings of your IaaS. Example: Check the security settings for your VM in Azure.</p>
<h3 id="packet-capture">Packet capture</h3>
<p><img src="https://imgur.com/ChIOGIv.gif" alt="" class="align-center" /></p>
<p>Investigate network by capturing the network packets. You can choose the Remote or local IP and port addresses.</p>
<p class="notice--info">👨🏫 In Exam remember: <strong>Most network diagnostics issues can be detected and analyzed with Azure Network Watcher</strong></p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<h2 id="become-full-stack-developer-">Become full stack developer 💻</h2>
<p class="notice--info">I teach at <a href="https://www.fullstackmaster.net">Fullstack Master</a>. If you want to become <strong>Software Developer</strong> and grow your carrier as new <strong>Software Engineer</strong> or <strong>Lead Developer/Architect</strong>. Consider subscribing to our full stack development training programs. You will learn <strong>Angular, RxJS, JavaScript, System Architecture</strong> and much more with lots of <strong>hands on coding</strong>. We have All-Access Monthly membership plans and you will get unlimited access to all of our <strong>video</strong> courses, <strong>slides</strong>, <strong>download source code</strong> & <strong>Monthly video calls</strong>.</p>
<ul>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/pro">All-Access Membership PRO plan</a></strong> to access <em>current</em> and <em>future</em> <strong>angular, node.js</strong> and related courses.</li>
<li>Please subscribe to <strong><a href="https://www.fullstackmaster.net/elite">All-Access Membership ELITE plan</a></strong> to get everything from PRO plan. Additionally, you will get access to a monthly <strong>live Q&A video call</strong> with <code class="language-plaintext highlighter-rouge">Rupesh</code> and you can ask <strong><em>doubts/questions</em></strong> and get more help, tips and tricks.</li>
</ul>
<p class="notice--warning">Your bright future is awaiting for you so visit today <a href="www.fullstackmaster.net">FullstackMaster</a> and allow me to help you to board on your dream software company as a new <strong>Software Developer, Architect or Lead Engineer</strong> role.</p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariAzure Network Watcher is a monitoring service with lots of other important services for network. Most network diagnostics issues can be detected and analyzed with Azure Network Watcher. Learn everything about Network Watcher in this article.Cloud Solution Architect Technical Interview Questions2021-11-19T00:00:00+00:002021-11-19T00:00:00+00:00https://www.rupeshtiwari.com/cloud-solution-architect-technical-interview-questions<blockquote>
<p>Are you willing to become cloud solution architect? I am going to add a few important technical questions that you must know for your next cloud solution architect interview. My answers would not be perfect so feel free to search for better answers. Important part from this article is what kind of technical questions you should know to become a good cloud solution architect. So focus on the questions that I am asking here and learn from anywhere about their answers.</p>
</blockquote>
<p>Most of the time you will be asked various open-ended technical questions. The best strategy to answer them is think of below 4 things:</p>
<ul>
<li>What is technology?</li>
<li>How does it work?</li>
<li>Why would you use it?</li>
<li>How do you architect them?</li>
</ul>
<p>You must know what a software Architect does?</p>
<ol>
<li>Business strategist</li>
<li>Advisor</li>
<li>designer</li>
</ol>
<p>They have to speak the languages of CEO, CIO, CFO and Engineers.</p>
<h2 id="what-is-amazon-ec2-elastic-compute-cloud-in-aws-or-virtual-machine-in-azure-cloud">What is Amazon EC2 (Elastic Compute Cloud) in AWS or Virtual Machine in Azure cloud?</h2>
<p>It is virtualization of computers on cloud. Amazon EC2 is a virtualized machine where we can install applications and OS. It is an Infrastructure as a Service offering. We virtualize storage, network, CPU etc. using hypervisors.</p>
<h2 id="what-is-caching-and-how-does-it-work">What is caching and how does it work?</h2>
<p>Caching is to offload servers by storing high frequently added information in the cache. If the user tries to take information from DB. It goes to cache and it will return. If the cache has no data it will go to the server return and cache it. Next person asks if it will return from the cache. It can improve performance, it can reduce latency. If you set up time to live of data in cache to timeout to keep it refreshing.</p>
<h2 id="what-is-dns-and-what-are-the-key-record-types">What is DNS? And what are the key record types?</h2>
<p>Domain name System used to map IP addresses to a user friendly name. Phonebook of the internet. You can route traffic Based on geography, latency , weight etc. Key Record Types are as following:</p>
<ol>
<li>A Record that match name to IPV4 address</li>
<li>AAAA Record that match name to IPV6 address</li>
<li>CNAME record where u map one domain to another domain</li>
<li>MX Record required for email.</li>
</ol>
<h2 id="what-is-the-mitm-man-in-the-middle-attack"> What is the MiTM (Man in The Middle) attack?</h2>
<p>“A Man-In-The-Middle attack is the type of attack where attackers intrude into an existing connection to intercept the exchanged data and inject false information. It involves eavesdropping on a connection and intercepting messages.” -Toolbox for IT.</p>
<p><img src="https://i.imgur.com/k2ofMxA.png" alt="" class="full" /></p>
<p>Hackers can use Packet sniffing software (WIRESHARK) to intrude the connection. Get the sender and receiver server information. He can get access to the website as Alice or Bob.</p>
<h2 id="data-encryption-at-rest-and-transit">Data encryption at rest and transit?</h2>
<p>At rest encrypt the disk and hence you get data encryption at rest out of the box. At transit use VPN or SSL/TLS (HTTPS = 443) to secure your data by hashing it during transit.</p>
<h2 id="what-is-symmetric-and-asymmetric">What is Symmetric and Asymmetric?</h2>
<p>Asymmetric: Sender data is encrypted with key1 and the receiver side uses key2 to decrypt the data.</p>
<p>Symmetric: both sender and receiver use the same key to encrypt and decrypt data.</p>
<h2 id="what-is-nmap">What is NMap?</h2>
<p>Used to scan a network, segment or host to find out the open ports. So that you can attack on those ports. Interviewer may ask if there is a linux machine and it has an application that is not working. What will you do. I will answer by saying I will first scan the network by using Nmap or wireshark to find out which ports are closed or opened etc.</p>
<h2 id="what-is-iam">What is IAM?</h2>
<p>Users are defined in IAM and they get assigned with roles and they have access based on the role to the protected resources. Also it does logging of each activity done by the user. Authentication & Authorization also done by IAM. Who is the user and what he is allowed to do.</p>
<h2 id="what-is-social-engineering">What is Social Engineering?</h2>
<p>Social Engineer is someone who is a master of asking seemingly non-invasive</p>
<p>or unimportant questions to gather information over time</p>
<ul>
<li>Gain trust</li>
<li>Reduce defenses</li>
</ul>
<p>Social Engineer use deception to get sensitive info from someone also s/he can combine number of techniques to gather sensitive information.</p>
<h2 id="how-would-you-secure-enterprise">How would you secure enterprise?</h2>
<p>Follow Multi layer approach:</p>
<ul>
<li>Policy</li>
<li>Look from User , Encryption, IAM and Tech perspective</li>
<li>Create Firewall protect your network perimeter</li>
<li>IDS and IPS system behind the firewall</li>
<li>DDoS ( Aws shield, cloud flare )</li>
<li>ACLS on the router and subnet</li>
<li>On the VM you install WAF to protect (XSS, XSRF)</li>
<li>Physical system secure ( like MAC address stop unused ports )</li>
<li>Phishing attack secure</li>
</ul>
<h2 id="how-would-you-secure-the-cloud">How would you secure the cloud?</h2>
<p>Follow Defence in Depth and secure physical to data layer.</p>
<p>Steps are as below:</p>
<ul>
<li>Physical security</li>
<li>IAM (Identity and Access Management)</li>
<li>DDos, Firewall (Perimeter), IDS/IPS or intelligent thread detection</li>
<li>NSG (Network Security Group): Network protection</li>
<li>WAF (Web Application Firewall): Compute, Patches, Malware protection etc.</li>
<li>Application: design/code review of applications installed</li>
<li>Data: User azure PaaS services which is by default data at rest.</li>
</ul>
<p><img src="https://i.imgur.com/HlQDoeU.png" alt="" class="full" /></p>
<h2 id="what-is-ddos-">What is DDoS ?</h2>
<p>Distributed denial of service (DDOS) is when a server is exhausted by serving large numbers malicious requests and could not serve genuine requests is called DDoS. It loses CIA principle ( Confidentiality , Integrity and Availability ). Now it is not available for genuine users. Azure DDoS protection , cloud flare , AWS shield can save you.</p>
<h2 id="what-is-ids-or-ips">What is IDS or IPS?</h2>
<p>IDS (IIntrusion Detection System)</p>
<ul>
<li>Been around for quite awhile, fairly common and easier to set up. Logs alerts and events for later analysis</li>
<li>Allows for reactive response / research</li>
</ul>
<p>IPS ( Intrusion Prevention System)</p>
<ul>
<li>Newer platform over the last few years</li>
<li>Enables prevention (such as blocking IP addresses, etc.)</li>
<li>False positives could block legitimate traffic</li>
</ul>
<h2 id="what-is-dhcp">What is DHCP?</h2>
<p>Dynamic host configuration protocol (DHCP) This is a server that assigns automatically/dynamically IP to connected hosts or servers. Suppose you have 300K systems then you use DHCP. How does it work? Whenever a new system comes up. It sends a broadcast (“Hey I am new in the network I need an address” ) called DHCP discover. All the DHCP servers will respond ( called as DHCP offer ). If there are 10 DHCP servers the client will receive 10 IPs then the client has to send a DHCP request to his favorite or desired DHCP server. Then the server sends back a DHCP acknowledgement to the client.</p>
<h2 id="what-is-a-vlan">What is a VLAN? </h2>
<p>VLAN (virtual local area network) You have 100 computers in one switch and within one subnet. Limit the finance to access the things only they need. Finance, HR, Sales, ITOps team might need access to everything. You can virtualize your switch to chop into multiple virtual logical switches. This way you can enhance security for those logical groups of computers. <a href="https://www.youtube.com/watch?v=R-4K5aHg-iw&list=PL0azhNeBK66KfW04TZBQWkX62hhnFcb9E&index=3&ab_channel=GoCloudArchitects">Cloud Architect Technical Interview (Master The Cloud Architect Interview Questions!)</a></p>
<h2 id="what-is-vlan-trunking">What is VLAN Trunking?</h2>
<p>4 VLAN in single switch and another 4 in other switch u run a single cable between them to establish connection. That is called VLAN Trunking? How do you do it in a private and secure manner? The trunk has a tag for a VLAN. Each VLAN will get its own IEEE 802.1Q tag.</p>
<p><img src="https://i.imgur.com/qbywToj.png" alt="" class="full" /></p>
<p>In a cloud computing environment. On-premise to Azure cloud when you do direct connection or express route. You send data over VLAN over a Trunk. Azure takes your information and your VLAN Tag and keeps it separate from other customers’ traffic.</p>
<h2 id="whats-the-difference-between-a-hybrid-cloud-and-a-full-cloud-environment--and-what-are-the-advantages-of-each">What’s the difference between a hybrid cloud and a full cloud environment ? And what are the advantages of each?</h2>
<p>Hybrid Cloud architecturally speaking you have your datacenter that has servers, compute, storage, networking and security and you also have the cloud. And you have both connected to get the benefits of your current infrastructure and connect the cloud to get the agility, scaling, and resilience.</p>
<p>Performance: You get lower latency from your datacenter compared to the cloud. Also you get control in case the cloud is down and you have your datacenter still running.</p>
<p><img src="https://i.imgur.com/OY5bEW8.png" alt="" class="full" /></p>
<p>Native Cloud has No maintenance switch, routing, cooling, heating, physical security etc. If you are a new company and you don’t have money for capex then you should use cloud. So you save money and only spend on OPEX. You get more speed and scaling, reliance.</p>
<h2 id="how-can-autoscaling-help-with-ddos-protection">How can autoscaling help with DDoS protection?</h2>
<p>Hackers control multiple computers and they send requests to your network and overwhelm your systems. If your system can handle 10K web requests and DDoS is giving 30K requests per system. Each of the systems had autoscaling enabled. Then automatically when the system will be over utilized (75% or more) cloud will add a new system and load balance them. So now you spawn up 10 diff systems and now you can handle 50K requests. So the DDoS attack was controlled by auto scaling.</p>
<h2 id="what-is-classful-ip-addressing-and-problems">What is Classful IP Addressing and problems?</h2>
<p>Classful IP addressing host counts:</p>
<ul>
<li>Class A - 16 million host identifiers</li>
<li>Class B - 65535 host identifiers</li>
<li>Class C - 254 host identifiers</li>
</ul>
<p>Classful IP addressing IP ranges:</p>
<ul>
<li>Class A 1.0.0.0-127.255.255.255 with mask 255.0.0.0 or (/8 in CIDR), 16 millions (2^24-2) IPs and 2^7 Networks.</li>
<li>Class B 128.0.0.0-191.255.255.255 with default subnet mask of 255.255.0.0 (/16 in CIDR) 65K (2^16-2) IPs and 2^14 Networks.</li>
<li>Class C 192.0.0.0-223.255.255.255 with default subnet mask of 255.255.255.0 (/24in CIDR) 254 (2^8-2) IPS and 2^21 Networks.</li>
<li>Class D and E are reserved for multicasts. 224.0.0.0 - 239.255.255.255</li>
</ul>
<p>The problem would commonly occur when an organization required more than 254 host machines and therefore would no longer fall into class C but rather class B. This means that the organization would use a class B license even though they had far less than 65,535 hosts. Therefore if an organization only required 2,500 hosts, they would be wasting about 63,000 hosts by holding a class B license which would greatly decrease the availability of IPv4 addresses unnecessarily</p>
<p>From <<a href="https://www.keycdn.com/support/what-is-cidr">https://www.keycdn.com/support/what-is-cidr</a>></p>
<p><img src="https://i.imgur.com/YxC6p45.png" alt="" class="full" /></p>
<h2 id="what-is-cidr">What is CIDR?</h2>
<p>Classless Inter-Domain Routing (CIDR) divides the IP address in network ID and host Id. CIDR is also known as supernetting. It is a method of allocating IP address and IP routing. It replaces the old school of classful IP addressing. CIDR is based on <a href="http://searchnetworking.techtarget.com/definition/variable-length-subnet-mask">variable-length subnet masking</a> (VLSM). This allows it to define prefixes of arbitrary lengths making it much more efficient than the old system. CIDR IP addresses are composed of two sets of numbers. The network address is written as a prefix, like you would see a normal IP address (e.g. 192.255.255.255). The second part is the suffix which indicates how many bits are in the entire address (e.g. /12). Putting it together, a CIDR IP address would look like the following: 192.255.255.255/12. The network prefix is also specified as part of the IP address. This varies depending upon the number of bits required. Therefore, taking the example above, we can say that the first 12 bits are the network part of the address while the last 20 bits are for host addresses.</p>
<h2 id="what-is-sqs">What is SQS?</h2>
<p>Simple Queueing System. Decouple A and B so you can scale both of them independently. It is used to decouple 2 independent components or microservices by giving reliable communication mechanisms.</p>
<h2 id="what-is-autoscaling">What is autoscaling?</h2>
<p>Buy the biggest server to meet your busiest day. On Christmas they get 10M requests. They can buy VMs and put them in an auto scaling group to scale automatically.</p>
<p>However in the cloud you get an autoscaling group for EC2 instances in AWS and you have Virtual Machine Scale Set in Microsoft Azure cloud.</p>
<p>There you just need to define:</p>
<ol>
<li>Desired Capacity : 2</li>
<li>Minimum Capacity : 2</li>
<li>Maximum Size: 10 when 75% of both machine is reach</li>
</ol>
<h2 id="what-is-a-load-balancer-why-do-you-use-it">What is a Load Balancer? Why do you use it?</h2>
<p>It is a server or a hardware that will determine which server to send traffic. To share the traffic. What if a single server failed because of load. You don’t have redundancy. For designing High Performance, High Available web applications, you need a load balancer. Load balancers increase the availability and performance of your system.</p>
<p>We can divide the load balancer in 2 parts: network and application load balancer.</p>
<p>Network Load Balancers are really fast because they work at layer -4. They are looking at TCP headers and they are pushing at your servers. So if you want high performance you need a network load balancer. Cross-Region or global load balancing.</p>
<p>Application Load Balancers work at layer-7 so it is a web-traffic load balancer so they enable us to manage traffic to your web application. SSL termination, cookie-based session, round robin for load-balancing traffic. If you want application intelligence and routing then use this.</p>
<p>If you want to route between microservices then use it.</p>
<p><img src="https://lh5.googleusercontent.com/KuonKIAH2IE99BhbpFE2Ry0vj3vhg3TFQ9TTpjm8UJc1WR8fS2BAm4gn1XcW90Dc_Y-VreU5_M2vqkkeAURU3dz76j16WcBiEkZqCRsYffYo6Ca4EA39ORPgGNxV7wmgoqBSV7oy=s0" alt="How load balancing works" /></p>
<h2 id="what-is-api-gateway-and-its-benefits">What is API Gateway and its benefits?</h2>
<p><a href="https://dashbird.io/blog/can-api-gateway-act-load-balancer/">https://dashbird.io/blog/can-api-gateway-act-load-balancer/</a></p>
<p>API Gateway offers below which ALB does not:</p>
<ul>
<li>It integrates with IAM</li>
<li>Authentication & Authorization</li>
<li>API Tokens issuing and managing</li>
<li>Throttling</li>
<li>Caching</li>
</ul>
<h2 id="what-is-firewall">What is Firewall?</h2>
<ul>
<li>It blocks traffic at the edge of your network,</li>
<li>it keeps your network secure</li>
<li>It blocks all traffic denied by policies</li>
<li>It lets your outbound traffic back if it’s coming from you.</li>
</ul>
<p>A firewall protects the perimeter of the edge of your network. It is a security appliance that’s basically standing guard saying you are not allowed in, you are allowed. It looks at traffic based on policy. By default it will block all the traffic from coming in. It will allow your internal traffic out on the internet and it will allow all your return traffic back through the firewall. Because Firewalls are Stateful therefore. Within perimeter if Mike wants to open google.com. Firewall is Stateful it will say okay Mike started the session, let his return traffic come back because, I know it’s destined for mike and I know it’s related to the same session where Mike requested google.com.
<img src="https://i.imgur.com/gpGJyXV.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/WPD9IiY.png" alt="" class="full" /></p>
<h2 id="how-does-cloud-computing-affect-an-organizations-costs">How does cloud computing affect an organization’s costs?</h2>
<p>Saves capital cost. Costs like Workstation, firewall, load balancer, cooling, networking, power etc.</p>
<p>Operation cost also like : Electric bill, Network, staff</p>
<p>Move to the cloud, nothing to buy. You can do lift and shift and save capex. You only pay for what you use. Your Capex goes down and Opex goes up. Agility benefit, u can spawn up vms fastly.</p>
<h2 id="what-are-vpc-flow-logs--nsg-flow-log-in-azure">What are VPC flow Logs ? NSG flow log In Azure?</h2>
<p>In networking we have Cisco NetFlow. It will examine the traffic, they would cross cisco routers and you could say, hm the traffic all going from here to end point B and check for security violation. You can find areas of congestion in your network. Check any issue on network traffic and troubleshoot.</p>
<p>Vpc flow log gives info about traffic flow. They are excellent for any network troubleshooting especially in security.</p>
<p>It will tell you below things:</p>
<ul>
<li>Source</li>
<li>Destination</li>
<li>Protocols</li>
<li>Port number</li>
</ul>
<p>Something accepted by ACL but rejected by security group. If this traffic was supposed to be a web app hosted in a VM and you want to know why it got rejected by SG. Check the VPC flow logs.</p>
<h2 id="whats-needed-for-a-simple-lift-and-shift-of-a-system-to-the-cloud">What’s needed for a simple lift and shift of a system to the cloud?</h2>
<p>Define the configuration of the server like How many cores, RAM and Storage.</p>
<h2 id="what-is-the-difference-between-latency-based-routing-and-geolocation-based-routing">What is the difference between Latency based Routing and Geolocation based routing?</h2>
<p>Latency routing is for speed and performance, geo-location routing will also optimize the speed and performance however it will also send traffic to appropriate websites based on their locations.</p>
<p>Latency based routing determines the lowest latency and sends you to the web server with the lowest latency so you get the best experience</p>
<p>Latency based routing done by Traffic Manager/Front door</p>
<p>Geolocation routing looks at the source IP address and determines what is closest to you and sends them to a different destination. Example you can check IP and decide province and route to that website. For Finch province route to the French website. Route the English province to the English website. If you are dealing with a company having Multi Language websites there, use this.</p>
<h2 id="what-is-the-difference-between-a-region-an-availability-zone-an-edge-location-and-local-zone">What is the difference between a region, an availability zone, an edge location and local zone?</h2>
<h3 id="what-is-region">What is Region?</h3>
<p>AWS has the concept of a Region, which is a physical location around the world where we cluster data centers. We call each group of logical data centers an Availability Zone. Each AWS Region consists of multiple, isolated, and physically separate AZs within a geographic area.</p>
<h3 id="what-is-availability-zone">What is Availability Zone?</h3>
<p>From <<a href="https://aws.amazon.com/about-aws/global-infrastructure/regions_az/">https://aws.amazon.com/about-aws/global-infrastructure/regions_az/</a>> An Availability Zone (AZ) is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region.</p>
<h3 id="what-is-local-zones">What is Local Zones?</h3>
<p>From <<a href="https://aws.amazon.com/about-aws/global-infrastructure/regions_az/">https://aws.amazon.com/about-aws/global-infrastructure/regions_az/</a>> <a href="https://aws.amazon.com/about-aws/global-infrastructure/localzones/">AWS Local Zones</a> place compute, storage, database, and other select AWS services closer to end-users. This is for critical latency systems like Games.</p>
<h3 id="what-is-edge-location">What is Edge Location?</h3>
<p>From <<a href="https://aws.amazon.com/about-aws/global-infrastructure/regions_az/">https://aws.amazon.com/about-aws/global-infrastructure/regions_az/</a>> Edge location is Reliable, low latency and high throughput network connectivity. Amazon CloudFront peers with thousands of Tier 1/2/3 telecom carriers globally, is well connected with all major access networks for optimal performance, and has hundreds of terabits of deployed capacity. CloudFront Edge locations are connected to the AWS Regions through the AWS network backbone - fully redundant, multiple 100GbE parallel fiber that circles the globe and links with tens of thousands of networks for improved origin fetches and dynamic content acceleration.</p>
<p>To deliver content to end users with lower latency, Amazon CloudFront uses a global network of 225+ Points of Presence (215+ Edge locations and 13 regional mid-tier caches) in 90 cities across 47 countries. Amazon CloudFront Edge locations are located in: North America, Europe, Asia etc.</p>
<p>From <<a href="https://aws.amazon.com/cloudfront/features/?whats-new-cloudfront.sort-by=item.additionalFields.postDateTime&whats-new-cloudfront.sort-order=desc">https://aws.amazon.com/cloudfront/features/?whats-new-cloudfront.sort-by=item.additionalFields.postDateTime&whats-new-cloudfront.sort-order=desc</a>></p>
<h2 id="what-is-iam-1">What is IAM?</h2>
<p>AAA (authentication, authorization and accounting)</p>
<p>Who is allowed in the system, What they can do and Track what they do.</p>
<h2 id="what-is-an-8021q-tag">What is an 802.1Q tag?</h2>
<p>VLAN is a subnet in the cloud. Take a physical switch and virtualize them here are examples:</p>
<ul>
<li>Port 10-20 vlan -2</li>
<li>Port 20-30 vlan -3</li>
<li>Port 30-40 vlan-4</li>
</ul>
<p>Take a single physical connection coming from on-premise to cloud vlan network using 802.1Q tag. With 802.1q tag aws separate your traffic from other customers for direct connection.</p>
<h2 id="what-is-cdn">What is CDN?</h2>
<p>Amazon CloudFront is a fast content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to customers globally with low latency, high transfer speeds, all within a developer-friendly environment.</p>
<p>From <<a href="https://aws.amazon.com/cloudfront/">https://aws.amazon.com/cloudfront/</a>></p>
<ul>
<li>Speed up content</li>
<li>Reduce network bandwidth to aws network</li>
<li>Closer to user</li>
<li>Guaranteed performance</li>
</ul>
<p>Instead of going to Amazon.com go to dns and data center. It goes to CDN and you are calling from Miami and serving the site from that location. If cdn has not site cached then it can go fastly to the datacenter and fetch the content since cdn is on aws edge location which is connected to the aws backbone network. Reducing load on web server and Transfer cost on AWS or any cloud network for other cloud service providers.</p>
<h2 id="what-is-the-process-to-take-a-web-server-and-make-it-publicly-available-on-the-internet-if-you-are-inside-of-the-aws-cloud-">What is the process to take a web server and make it publicly available on the internet if you are inside of the AWS cloud ?</h2>
<p>You will take an external public IP from amazon pool and assign it to EC2 instance. From EC2 now you want the default router to be the internet router. Therefore, you need an internet gateway.</p>
<h2 id="what-is-the-optimal-subnet-size-for-a-point-to-point-wan-link">What is the optimal subnet size for a point-to-point WAN link?</h2>
<p>Answer is /30 So I got 4 IP 2 reserved and got 2 free.</p>
<ul>
<li>192.168.0.0/30</li>
<li>192.168.0.1/30</li>
<li>192.168.0.2/30</li>
<li>192.168.0.3/30</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">0.0.0.0</code> reserved for internet and <code class="language-plaintext highlighter-rouge">255.255.255.255</code> is for broadcast</p>
<h2 id="what-is-the-difference-between-iops-and-throughput">What is the difference between IOPS and throughput?</h2>
<p><strong>IOPS</strong> (input output per seconds) determines how frequently you can read and write to the disc and how fast. IOPS is related to latency the higher the IOPS, meaning the more read/write operations per second, the lower the latency.</p>
<p><strong>NVME</strong> (Non-Volatile Memory Express) is faster than SSD. Both NVME and SSD drives tend to have relatively low latency because the read/write operations are very fast or the IOPS are very fast. Magnetic drives have much higher latency and much lower IOPS. But if you put a bunch of Magnetic drives together in a RAID environment you can still get great THROUGHOUT. IOPS inversely proposal to Latency</p>
<p>The higher IOPS the lower latency and the latency on the network and vice versa. THROUGHPUT is the amount of stuff and data that can be moved at any one period of time.</p>
<p>Example: cars run 55 mph then its throughput would be whatever it could stuff in the trunk, or the boot, But if you had a Freight Train with 55 mph then you could carry a lot more stuff. And the stuff is the THROUGHPUT.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IOPS = LATENCY
THROUGHPUT = The amount of STUFF
</code></pre></div></div>
<p>Use case of IOPS and Throughput:</p>
<p>Video editors need a drive that needs high throughput because they are working with large video files but they can tolerate the latency.</p>
<p>A database needs extreme speed in terms of read and write operation but it is not moving large amounts of data then it needs higher IOPS.</p>
<h2 id="what-is-the-difference-between-acl-and-nsg-">What is the difference between ACL and NSG ?</h2>
<p>Access Control Lists (ACL) is a Networking concept. It keeps traffic out of the subnet and Network Security Group (NSG) keeps traffic out of the Host. ACL protects the subnet and NSG protects the server. ACLs are stateless meaning it is not tracking the state of the connection. The traffic that is allowed in is not being monitored, so therefore, you don’t know a lot of return traffic. So rules in ACL have to be applied in both directions.</p>
<p>NSG is just like a firewall, it is stateful so you have to only enable the inbound direction. So if traffic is coming to the host then the return traffic is allowed from the same host back to the source.</p>
<p>ACL works at layer -3, NSG is stateful They are stateful, meaning that they allow return traffic to flow. In general allow everything in and out and block the specific one.</p>
<h2 id="stateful-vs-stateless">Stateful vs Stateless?</h2>
<p>Stateful services keep track of sessions or transactions and react differently to the same inputs based on that history. Stateless services rely on clients to maintain sessions and center around operations that manipulate resources, rather than the state.</p>
<ol>
<li>ACL is stateless</li>
<li>Firewall is stateful</li>
<li>NSG is stateful both in AWS and Azure</li>
</ol>
<h2 id="what-is-raid-0-1-5--10">What is RAID 0, 1, 5 , 10?</h2>
<p>Redundant Array of Independent Disks (RAID) is a virtual disk technology that combines multiple physical drives into one unit.</p>
<ul>
<li><strong>Raid-0</strong>: Stripping, high speed, no FT, 100% capacity, data can not be recovered</li>
<li><strong>Raid-1</strong>: Mirroring , normal speed, FT, 50% capacity, data can be recovered when 1 disk crash, during 1 disk crash performance remains same, minimum 2 disk required. If you have 2 , 10TB drives you get only 1TB of storage.</li>
<li><strong>Raid-5</strong>: Stripping with Parity, High Throughput and normal IOPS, normal speed, FT, 1 disk capacity we lose, minimum 3 disk we need, 1 disk for parity bit, if any 1 disk failed data can be recovered, during disk failure performance become more down because it has to calculate the parity bit, capacity wise 1 disk capacity lost and used for parity saving</li>
<li><strong>Raid-10</strong>: Stripping and Mirroring together, logical drive made of raid 0 and raid 1, first we create 2 raid 0 disks (4) then we combine them to a logical drive called raid 1. finally it becomes raid 10. This gives high performance, FT, and capacity is 50% when 2 disks are lost.</li>
</ul>
<p><strong>Stripping</strong> means distributing bits in different disk and hence faster write because per disk you write very less bit and parallely you write so your write and read both are faster. Here you get speed however you don’t get Fault Tolerance one disk fails then entire data is lost.</p>
<p><strong>Mirroring</strong> means duplicating means while writing you write all bits simultaneously in 2 different disks to keep redundant disk and protect from data loss. Mirroring does not slow down read write since its writing parallely in 2 disks together. However, since you are writing all bits in one disk at a time, it is slower than the Stripping technique. Here you do not get stripping technology disk speed however you get fault tolerance so if one disk failed then you still can recover your data since you had other disk available as redundant. You have to pay more for mirroring since you want to store 1 TB then you need 2TB for redundancy.</p>
<h2 id="what-is-the-aws-shared-responsibility-model">What is the AWS Shared Responsibility Model?</h2>
<p>We take care of the cloud. You take care in the cloud.</p>
<p>AWS can manage below:</p>
<ul>
<li>Physical network</li>
<li>Hardware</li>
<li>Manage server for SERVERLESS</li>
</ul>
<p><img src="https://i.imgur.com/2RQxYa9.png" alt="" class="full" /></p>
<p><a href="https://aws.amazon.com/compliance/shared-responsibility-model/">https://aws.amazon.com/compliance/shared-responsibility-model/</a></p>
<h2 id="how-do-you-secure-vpc">How do you secure VPC?</h2>
<p>Follow Defense in Depth model and secure from physical to data layer.</p>
<h3 id="physical">Physical</h3>
<p>1- MAC are open for multiple unwanted ports then block them</p>
<h3 id="perimeter">Perimeter</h3>
<ol>
<li>Firewall</li>
<li>IPS</li>
<li>IDS</li>
<li>DDOS</li>
</ol>
<h3 id="iam">IAM</h3>
<ol>
<li>Single sign On and Identity Providers</li>
</ol>
<h3 id="network">Network</h3>
<ol>
<li>Monitoring</li>
<li>Encryption</li>
<li>Session protection (XSS, XSR, Session Fixation, Session Side Jacking)</li>
<li>Network segmentation</li>
<li>Inside VPC to keep unwanted traffic out of the subnet use ACL</li>
</ol>
<h3 id="compute">Compute</h3>
<ol>
<li>NSG to secure host</li>
<li>Anti-virus</li>
<li>Single Sign On</li>
<li>Managed Identity</li>
<li>Virtual Patching (this is for legacy apps lift and shift with the bad code: WAS and WAF whenever web application scan finds a threat it creates a automatic rule in the web application firewall )</li>
</ol>
<h3 id="applications">Applications</h3>
<ol>
<li>SSL/TLS, HTTPS</li>
<li>Single Sign ON</li>
<li>Application integrity ( adhering to policies )</li>
<li>Vulnerability Scans</li>
</ol>
<h3 id="data">DATA</h3>
<p>Server less -> enable the threat detection -> SEIM , enable data encryption at rest and over network .</p>
<ol>
<li>Encryption at rest and transit</li>
<li>Masking</li>
</ol>
<h2 id="what-are-the-options-for-using-the-cloud-for-disaster-recovery">What are the options for using the cloud for disaster recovery?</h2>
<table>
<thead>
<tr>
<th>#</th>
<th>Plan</th>
<th>Cost</th>
<th>Speed</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Manual Backup and Restore, <strong>Point in time backups into the DR region.</strong></td>
<td><strong>Cheap($)</strong></td>
<td><strong>RPO = Hours, RTO = Days</strong>. Long time to come back to service.</td>
</tr>
<tr>
<td>2</td>
<td><strong>Pilot Light (RPO 1min RTO 1hr)</strong>. VMs are created on the DR site and turned off. However the database in the DR site is in active mode and you write your data in DR site live. WEB and Business VMs will have applications and configuration will be copied to DR sites however they are turned off.</td>
<td><strong>$$</strong></td>
<td><strong>RPO=Minutes, RTO=Hours</strong>.</td>
</tr>
<tr>
<td>3</td>
<td><strong>WARM standby(RPO 1sec, RTO 1Min)</strong>. Replicate your environment but use very small instances in the DR site but place them in an auto scaling group. When DR workloads are scaled up fully then it is called HOT standby. The more scale up the RTO you get in the DR site.</td>
<td><strong>$$$</strong></td>
<td><strong>RPO = Seconds, RTO = Minutes</strong></td>
</tr>
<tr>
<td>4</td>
<td><strong>Multi Region (Multi Site) Active-Active (RPO and RTO are zero)</strong> Your workload is deployed to, and actively serving traffic from, multiple AWS Regions. This strategy requires you to synchronize data across Regions. Use DNS to route the traffic</td>
<td><strong>Most Costly</strong></td>
<td><strong>RPO=0, RTO=0</strong></td>
</tr>
</tbody>
</table>
<h2 id="when-vpn-and-direct-connection">When VPN and Direct connection?</h2>
<p>If you need guaranty consistent latency and guaranty bandwidth use Direct connection since it is connected with direct wire. Costly option. Use the VPN because it is cheaper, works with the internet, and is flexible since everyone has the internet. You can create connections on demand. It was easy to connect to multiple remote sites. The side is you depend on internet bandwidth.</p>
<h2 id="you-have-site-where-u-have-10-remote-campuses-the-main-site-is-hosted-in-the-cloud-everyone-wants-to-talk-to-everyone-how-can-you-set-up-your-topology">You have site where u have 10 remote campuses. The main site is hosted in the cloud. Everyone wants to talk to everyone. How can you set up your topology? </h2>
<table>
<thead>
<tr>
<th>#</th>
<th>Technique</th>
<th>Description</th>
<th>Speed</th>
<th>Maintenance</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td><strong>Full Mesh</strong></td>
<td>Number of edges will be n x (n-1)/2</td>
<td>FAST</td>
<td>High</td>
</tr>
<tr>
<td>2</td>
<td><strong>VPN</strong></td>
<td>Connect on-premise with site2site VPN and connect each on-premise.</td>
<td>Normal</td>
<td>Medium</td>
</tr>
<tr>
<td>3</td>
<td><strong>Hub & Spoke (Cloud Hub)</strong></td>
<td>Hub and spoke topology on cloud</td>
<td>Normal</td>
<td>Low</td>
</tr>
<tr>
<td>4</td>
<td><strong>Transit Gateway</strong></td>
<td>Managed service from cloud provider to provide transit connection among virtual networks in cloud.</td>
<td>FAST</td>
<td>High</td>
</tr>
</tbody>
</table>
<h2 id="what-function-is-achieved-with-ipsec">What function is achieved with IPSEC?</h2>
<p>IPSec provides the ability to authenticate each remote end to prevent man-in-the-middle attacks. And it ensures the integrity of the data because it uses a hashing algorithm to encrypt the payload. IPSec provides Non-Repudiation means Assurance that the sender of information is provided with proof of delivery and the recipient is provided with proof of the sender’s identity, so neither can later deny having processed the information. Also it provides the tunnel for private IP addresses and private traffic and private routing information over public networks.</p>
<p>From <<a href="https://csrc.nist.gov/glossary/term/non_repudiation">https://csrc.nist.gov/glossary/term/non_repudiation</a>> It is a replacement of internet protocol. IPsec is a suite of protocols. IP protocol and security protocol. It encrypts the payload and keeps the header un-encrypted. It is used in VPN to create a tunnel of encryption from on-premise to cloud.</p>
<h2 id="how-can-you-scale-relational-databases">How can you scale relational databases?</h2>
<p>Ask the question what type of traffic is being used by your database?</p>
<ul>
<li>It’s a combination of read and write traffic.</li>
</ul>
<p>If you have read traffic more then create read replicas and offload the read work to read replicas. Which will free up resources on the master database. You can further reduce the load on read replicas by adding some caching. In order to reduce write load, so you don’t have peaks and valleys you can use queuing system.</p>
<h2 id="acid-vs-base-">ACID vs BASE ?</h2>
<p>Relational DB is ACID by nature:</p>
<ol>
<li>Atomicity</li>
<li>Consistency</li>
<li>Isolation</li>
<li>Durability</li>
</ol>
<p>NoSQL are BASE by nature:</p>
<ol>
<li>Basic Availability: Application must handle partial failures itself</li>
<li>Soft State: DB state can change even without inputs</li>
<li>Eventually Consistency: • DB will “eventually” become consistent</li>
</ol>
<h2 id="what-is-virtualization-how-does-it-work-and-why-will-you-use-it">What is virtualization, how does it work and why will you use it?</h2>
<p>Take a bare metal server, add a hypervisor and create a virtual machine on the hypervisor.</p>
<p><img src="https://i.imgur.com/GvUVlj3.png" alt="" class="full" /></p>
<h2 id="block-storage-and-object-storage-how-they-work-what-are-their-advantages">Block Storage and Object Storage how they work, what are their advantages?</h2>
<p>If you need storage on a VM that does not go away with reboot, you are going to use something like block storage. If you have a static website or you want to distribute software, you are going to use something like object storage. If you are going to create data-lake , use object storage. If you need high performance storage you will use block storage. Object storage is cheap and can store large volumes of data.</p>
<h3 id="object-storage">Object Storage</h3>
<p>Object in the object storage is consist of :</p>
<ul>
<li>Data</li>
<li>Unique Id: 128 bit</li>
<li>Expandable metadata: contextual data</li>
</ul>
<p><img src="https://i.imgur.com/uG9uip3.png" alt="" class="full" />
<img src="https://i.imgur.com/chlbywt.png" alt="" class="full" /></p>
<p>Objects can’t be modified—you have to write the object completely at once. Object storage also doesn’t work well with traditional databases, because writing objects is a slow process and writing an app to use an object storage API isn’t as simple as using file storage.</p>
<p>From <<a href="https://www.redhat.com/en/topics/data-storage/file-block-object-storage">https://www.redhat.com/en/topics/data-storage/file-block-object-storage</a>></p>
<h3 id="block-storage">Block Storage</h3>
<p>Block storage chops data into blocks—get it?—and stores them as separate pieces. Each block of data is given a unique identifier, which allows a storage system to place the smaller pieces of data wherever is most convenient. That means that some data can be stored in a <a href="https://www.redhat.com/en/topics/linux">Linux®</a> environment and some can be stored in a Windows unit. It is usually deployed in storage-area network (SAN) environments and must be tied to a functioning server. Block storage can be expensive. It has limited capability to handle metadata, which means it needs to be dealt with at the application or database level—adding another thing for a developer or systems administrator to worry about. From <<a href="https://www.redhat.com/en/topics/data-storage/file-block-object-storage">https://www.redhat.com/en/topics/data-storage/file-block-object-storage</a>></p>
<p><img src="https://i.imgur.com/xpB53D7.png" alt="" class="full" /></p>
<h2 id="what-is-nat-and-why-would-you-use-it">What Is NAT and Why would you use it?</h2>
<p>Network address translation and its really about translating one IP address to another address.</p>
<p>Now they could be used to connect private addresses to the internet by translating it into a public address.</p>
<ul>
<li><a href="https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/0030_Enable_a_Server_on_the_Inside_Network_to_Reach_the_Internet_Using_a_Public_IP_address">Enable a Server on the Inside Network to Reach the Internet Using a Public IP address</a></li>
<li><a href="https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/0040_Enable_Users_on_the_Inside_Network_to_Access_the_Internet_Using_the_Outside_Interface's_Public_IP_Address">Enable Users on the Inside Network to Access the Internet Using the Outside Interface’s Public IP Address</a></li>
<li><a href="https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/0050_Make_a_Server_on_the_Inside_Network_Available_on_a_Specific_Port_of_a_Public_IP_Address">Make a Server on the Inside Network Available on a Specific Port of a Public IP Address</a></li>
<li><a href="https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/0060_Translate_a_Range_of_Private_IP_Addresses_to_a_Range_of_Public_IP_Addresses">Translate a Range of Private IP Addresses to a Range of Public IP Addresses</a></li>
</ul>
<p>From <<a href="https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/Common_Use_Cases_for_NAT">https://docs.defenseorchestrator.com/Configuration_Guides/Security_Policy_Management/Network_Address_Translation/Common_Use_Cases_for_NAT</a>></p>
<h2 id="what-is-stateful-firewall-or-aws-nsg">What is Stateful Firewall or AWS NSG?</h2>
<p>Stateful means watching the state of the connection, they keep the context of the route and remember which request is initiated from which host. If you dis-allowed outbound to port 80 then automatically inbound flights are prohibited.</p>
<p>When you initiate a connection it goes through the firewall on the way out to the internet. The firewall looks at your connection and it tracks what you have done. Because when a response comes it will allow you to back in because it knows you initiated the connections.</p>
<h2 id="when-connecting-to-aws-when-would-you-use-direct-or-vpn-connection">When connecting to AWS when would you use direct or VPN connection?</h2>
<p>Direct connection is a wired connection from your on-premise route to Cloud provider router. You can use either use your ISP or direct Microsoft to connect direct connections. If you want guaranty bandwidth, performance and latency use direct connect.</p>
<p>VPN is You connect to the internet on both side and then you create a tunnel and you encrypt your data over the internet. So the internet Is not private and that’s why you have to encrypt your data. When you use a VPN it is cheaper because you are not buying a direct connection. You are connecting to the internet. VPN are really simple to setup because you can basically you can create any connection to any place that has internet connectivity, which is wonderful.</p>
<h3 id="advantages-of-vpn">Advantages of VPN</h3>
<ul>
<li>Econmical</li>
<li>Flexible</li>
<li>Easy to setup</li>
</ul>
<h3 id="disadvantages-of-vpn">Disadvantages of VPN</h3>
<ul>
<li>Internet bandwidth and performance is not guarantied. Your ISP may guarantee you access to them but when you are on their network and off their network and on the internet there are no guarantees.</li>
<li>Internet is called best effort delivered if you try to reach webpage it does mean its going to get there. It should, it will try but it will not be guarantied.</li>
<li>So internet speed is high you get good performance if low then you get bad performance.</li>
<li>Latency is not guaranty in direct connect you get 3 ms to reach. In VPN message1 can take 2 ms. Message2 takes 100 ms and then message3 takes 5 ms. So message2 can arrive after message3 and that may be a horrible situation. Message2 takes</li>
</ul>
<p><img src="https://i.imgur.com/uVEAiQQ.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/hFY5zPj.png" alt="" class="full" /></p>
<p>In reality, no physical tunnel exists, of course; the data has to travel through the same wires as any other data passing through the public network. Rather, VPN tunneling employs the concepts known as data encapsulation and encryption to safely carry data traffic through the non-secure environment. Encapsulation insulates the packet of data from other data traveling over the same network, while encryption makes the data “invisible” (unreadable) even to surveillance agents and criminals who recognize it as encrypted information. So it’s like the data is traveling inside a tunnel. Figure 1.0 below might help you visualize the process.</p>
<p>You can think of VPN tunneling as similar to the process of moving physical cash from one bank location to another using an armored transport van along public highways. The cash in this instance is your data, the public highway is the non-secure public network, and the armored van is analogous to a VPN tunnel (data encapsulation and encryption).</p>
<p>From <<a href="https://www.vpnmentor.com/blog/ultimate-guide-to-vpn-tunneling/">https://www.vpnmentor.com/blog/ultimate-guide-to-vpn-tunneling/</a>></p>
<h2 id="what-is-bgp-and-why-is-it-used">What is BGP and why is it used?</h2>
<p>Border Gateway Protocol (BGP) is layer 4 protocol. It is an exterior gateway protocol and it works with a path vector routing protocol that operates on TCP port 179. When you connect an entity to an external entity, that’s when you use an exterior gateway. That’s why organization use BGP to connect to AWS or GCP as opposed to something like OSPF (Open Shortest Path First, small routing within campus networks) or EIGRP (enhanced interior gateway routing protocol, used in networks of large commercial companies ) <a href="https://networkjourney.com/difference-between-eigrp-ospf-vs-bgp-routing-protocols/">https://networkjourney.com/difference-between-eigrp-ospf-vs-bgp-routing-protocols/</a></p>
<p>BGP is highly Tunable and highly scalable. For example if an internet routing table has basically 3 quarters of a million routes and BGP can easily handle that, whereas an interior gateway routing protocol could not. That’s why organizations use BGP to connect to external entities like AWS.</p>
<p>BGP Routing preferences sequence:</p>
<ol>
<li>Largest weight</li>
<li>Largest local preference</li>
<li>Shortest number of autonomous system hops</li>
<li>Lowest Med (metric)</li>
</ol>
<p>In order to select the next hop BGP checks who has the maximum weight that route it will take, next it will prefer maximum local preference, next shortest number of system hops, next lowest metric. This way it optimizes the route selection and data transfer over the internet.</p>
<h2 id="you-are-designing-a-video-streaming-software-what-should-you-use-tcp-or-udp-for-which-and-why">You are designing a video streaming software. What should you use TCP or UDP for? Which and why?</h2>
<p>It must be UDP. TCP is used for reliable transport. UDP is used for Realtime transport. Why? If I send this “Rupesh like design” via UDP and I receive “Rupesh Design” I lost like If I send this “Rupesh like design” via TCP and I receive “Rupesh Design Like” I lost, then TCP will retransmit the Like and you get the data in different order. For video streaming you do not want this behavior. All streaming things must be UDP! Because it is better, faster for these applications. There are no sliding windows. Performance is going to be what it’s going to be and there is no re-transmission on lost data.</p>
<h2 id="what-is-cxo-and-what-does-it-mean-to-present-differently-to-a-cxo-vs-an-engineer">What is CXO and what does it mean to present differently to a CXO vs an Engineer?</h2>
<p>CXO means a C-Level executive:</p>
<ul>
<li>COO</li>
<li>CFO</li>
<li>CEO</li>
<li>CIO (chief information officer)</li>
</ul>
<p>CXO means a C-level executive. They are extremely busy people. Because they are so busy they have an attention span of a few seconds at max. So you got to get your concept to the executive out to the point.</p>
<p>And you must talk about things that they care about.</p>
<p>The CEO is tasked with the organization’s strategy and increasing shareholder value meaning revenue growth or profitability growth.</p>
<p>The CFO cares about the organization’s finances. They are the gatekeeper for the organization’s money. When you are presenting to the CFO, you better be really good at basically doing some ROI (return on investment) modeling and showing that the value provided by your solution, provides greater value and savings or profitability to the company than it’s cost. You have to do that.</p>
<p>The CIO wants to know your technology solution is going to meet the CEO’s goals and needs. So you have to present this and it’s going to work. Now when you are dealing with the Engineers, they might need a lot of technical depth.</p>
<p>Presenting to the executive should be “SHORT to the POINT and the Solution”</p>
<p>Presenting to the engineer should be to dive deep and present how it works technically.</p>
<p>In most companies, the board of directors and the founders are at the top of the corporate hierarchy followed by the C-level executives namely the CEO, COO, CFO, etc. … Usually Vice Presidents (VPs) and Senior Vice Presidents (SVPs) report to C-level executives. <a href="https://resources.workable.com/hr-terms/c-level-executive">https://resources.workable.com/hr-terms/c-level-executive</a></p>
<h2 id="what-does-organization-strategy-mean">What does organization strategy mean?</h2>
<p>An organizational strategy is a plan that specifies how your business will allocate resources (e.g., money, labor, and inventory) to support infrastructure, production, marketing, inventory, and other business activities.</p>
<p><a href="https://getsling.com/blog/organizational-strategy">https://getsling.com/blog/organizational-strategy</a></p>
<h2 id="what-is-a-shareholder">What is a Shareholder?</h2>
<p>A shareholder is a party that legally owns shares of a company’s stock. Shareholders who own less than 50% of a company’s stock are known as ‘minority shareholders’, whereas shareholders who own 50% or more of a company’s stock are called ‘majority shareholders’. Shareholders are owners of the company, but they are not liable for the company’s debts.</p>
<p>From <<a href="https://www.investopedia.com/ask/answers/08/difference-between-a-shareholder-and-a-stakeholder.asp">https://www.investopedia.com/ask/answers/08/difference-between-a-shareholder-and-a-stakeholder.asp</a>></p>
<p>From <<a href="https://sumup.co.uk/invoices/dictionary/shareholders/">https://sumup.co.uk/invoices/dictionary/shareholders/</a>></p>
<h2 id="what-is-a-stakeholder">What is a stakeholder?</h2>
<p>Stakeholders can be:</p>
<ul>
<li>Owners and shareholders</li>
<li>Employees of the company</li>
<li><a href="https://www.investopedia.com/terms/b/bondholder.asp">Bondholders</a> who own company-issued debt</li>
<li>Customers who may rely on the company to provide a particular good or service</li>
<li>Suppliers and vendors who may rely on the company to provide a consistent revenue stream</li>
</ul>
<p>From <<a href="https://www.investopedia.com/ask/answers/08/difference-between-a-shareholder-and-a-stakeholder.asp">https://www.investopedia.com/ask/answers/08/difference-between-a-shareholder-and-a-stakeholder.asp</a>> A shareholder can sell their stock and buy different stock; they do not have a long-term need for the company. Stakeholders, however, are bound to the company for a longer term and for reasons of greater need. For example, if a company is performing poorly financially, the vendors in that company’s supply chain might suffer if the company no longer uses their services. Similarly, employees of the company, who are stakeholders and rely on it for income, might lose their jobs.</p>
<h2 id="what-is-aws">What is AWS?</h2>
<p>Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.</p>
<p>From <<a href="https://aws.amazon.com/what-is-aws/?nc1=f_cc">https://aws.amazon.com/what-is-aws/?nc1=f_cc</a>></p>
<h2 id="what-is-cloud-computing">What is cloud computing?</h2>
<p>Cloud computing is the on-demand delivery of IT resources over the Internet with pay-as-you-go pricing. Instead of buying, owning, and maintaining physical data centers and servers, you can access technology services, such as computing power, storage, and databases, on an as-needed basis from a cloud provider like Amazon Web Services (AWS).</p>
<p>From <<a href="https://aws.amazon.com/what-is-cloud-computing/">https://aws.amazon.com/what-is-cloud-computing/</a>></p>
<h2 id="cloud-computing-models">Cloud Computing Models?</h2>
<p>From <<a href="https://docs.aws.amazon.com/whitepapers/latest/aws-overview/types-of-cloud-computing.html">https://docs.aws.amazon.com/whitepapers/latest/aws-overview/types-of-cloud-computing.html</a>></p>
<h3 id="infrastructure-as-a-service-iaas">Infrastructure as a Service (IaaS)</h3>
<p>IaaS contains the basic building blocks for cloud IT. It typically provides access to networking features, computers (virtual or on dedicated hardware), and data storage space.</p>
<p>From <<a href="https://aws.amazon.com/what-is-cloud-computing/">https://aws.amazon.com/what-is-cloud-computing/</a>></p>
<h3 id="platform-as-a-service-paas">Platform as a Service (PaaS)</h3>
<p>PaaS removes the need for you to manage underlying infrastructure (usually hardware and operating systems), and allows you to focus on the deployment and management of your applications</p>
<p>From <<a href="https://aws.amazon.com/what-is-cloud-computing/">https://aws.amazon.com/what-is-cloud-computing/</a>></p>
<h3 id="software-as-a-service-saas">Software as a Service (SaaS)</h3>
<p>SaaS provides you with a complete product that is run and managed by the service provider.</p>
<p>From <<a href="https://aws.amazon.com/what-is-cloud-computing/">https://aws.amazon.com/what-is-cloud-computing/</a>></p>
<h2 id="six-advantages-of-cloud-computing">Six Advantages of Cloud Computing</h2>
<p>From <<a href="https://docs.aws.amazon.com/whitepapers/latest/aws-overview/six-advantages-of-cloud-computing.html">https://docs.aws.amazon.com/whitepapers/latest/aws-overview/six-advantages-of-cloud-computing.html</a>></p>
<ol>
<li>Go global in minutes</li>
<li>Trade capital expense for variable expense</li>
<li>Benefit from massive economies of scale</li>
<li>Increase speed and agility</li>
<li>Stop spending money running and maintaining data centers</li>
<li>Stop guessing capacity</li>
</ol>
<h2 id="cloud-computing-deployment-models">Cloud Computing Deployment Models?</h2>
<ul>
<li>Cloud</li>
<li>Hybrid</li>
<li>On Premise</li>
</ul>
<h2 id="what-is-region-1">What is Region?</h2>
<p>An AWS Region is a physical location in the world where we have multiple Availability Zones. Each Amazon Region is designed to be completely isolated from the other AWS Regions. This achieves the greatest possible fault tolerance and stability. AWS has 25 AWS regions with 240 countries.</p>
<h2 id="what-is-the-availability-zone">What is the Availability Zone?</h2>
<p>Availability Zones consist of one or more discrete data centers, each with redundant power, networking, and connectivity, housed in separate facilities. These Availability Zones offer you the ability to operate production applications and databases that are more highly available, fault tolerant, and scalable than would be possible from a single data center. Each Availability Zone is designed as an independent failure zone. This means that Availability Zones are physically separated within a typical metropolitan region and are located in lower risk flood plains (specific flood zone categorization varies by AWS Region).</p>
<h2 id="what-is-nas">What is NAS?</h2>
<p>Network attached storage. Centralize storage device for storing data on a network. I will have multiple hard drives in a RAID configuration. Used in home and small side business. It has a NIC so that you can connect this with a switch or router to be in the network and then you can access it from laptop, desktop mobile etc. It can be accessed as a shared drive. It has a single point of failure, if power supply fail in the NAS then other devices will not able to access it.
<img src="https://i.imgur.com/ij6H8IB.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/csRF4G2.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/KHdXMeJ.png" alt="" class="full" /></p>
<h2 id="what-is-san">What is SAN?</h2>
<p>Storage area network is special. SANs are a highly scalable, highly redundant and high speed network that stores and provides access to large amounts of data. SAN is fault tolerant since data is shared among several disk arrays. If a switch, disk array or server goes down data can still be accessed. Server accesses this data as if it was a local attached hard drive. In NAS it is recognized as a network drive. All other devices in SAN are connected to each other using fiber channel (fiber optics ) 2 -128 gbps speed. Fiber channel is very costly. The other option is iSCSI (Internet Small Computer System Interface) which is a cheaper alternative to using Fiber channel but they are not very fast. SANs are not affected by network traffic happening in LAN because SANs are not part of LAN. SANs are partitioned off, it’s basically a network all by itself. SANs are very expensive therefore, only large company, enterprises can afford this.</p>
<p>Some important slides from <a href="https://www.youtube.com/watch?v=3yZDDr0JKVc&ab_channel=PowerCertAnimatedVideos">NAS vs SAN - Network Attached Storage vs Storage Area Network</a></p>
<p><img src="https://i.imgur.com/DJUbzlf.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/M0Ot4YC.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/tzn8Bdy.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/U01CykX.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/xYXWhmt.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/LkshL3g.png" alt="" class="full" /></p>
<h2 id="what-is-scalability">What is Scalability?</h2>
<p>Scalability is the ability of a system to increase workload by adding resources. Scale up, down, out or in. <a href="https://www.youtube.com/watch?v=e8F2ls3n_Wo&ab_channel=zeisys"> Scalability vs Elasticity in 99 seconds</a></p>
<p><img src="https://i.imgur.com/VoFA3Eg.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/gZCASSS.png" alt="" class="full" /></p>
<h2 id="what-is-elasticity">What is Elasticity?</h2>
<p>Elasticity is the ability to automatically match the resources sufficient for a given workload size.<a href="https://www.youtube.com/watch?v=e8F2ls3n_Wo&ab_channel=zeisys">Scalability vs Elasticity in 99 seconds</a></p>
<p><img src="https://i.imgur.com/sbLUG3n.png" alt="" class="full" /></p>
<h2 id="network-load-balancer-vs-application-load-balancer">Network load balancer vs Application load balancer</h2>
<p>Application load balancer:</p>
<ul>
<li>URL routing can be done in</li>
<li>In level -7 (application)</li>
<li>Low performance</li>
<li>Load balance frontend servers</li>
</ul>
<p>Network load balancer:</p>
<ul>
<li>In level -4 (transport)</li>
<li>High performance</li>
<li>Load balance backend servers</li>
</ul>
<p>I will publish one more article where I will explain what are the soft skills you need to be successful in your next cloud architect interview so stay tuned.</p>
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.youtube.com/playlist?list=PL0azhNeBK66KfW04TZBQWkX62hhnFcb9E">Go Cloud Videos</a></li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariAre you willing to become cloud solution architect? I am going to add a few important technical questions that you must know for your next cloud solution architect interview. My answers would not be perfect so feel free to search for better answers. Important part from this article is what kind of technical questions you should know to become a good cloud solution architect. So focus on the questions that I am asking here and learn from anywhere about their answers.Getting Started with Monorepo with Nx Nrwl2021-11-19T00:00:00+00:002021-11-19T00:00:00+00:00https://www.rupeshtiwari.com/getting-started-with-monorepo-with-nx-nrwl<blockquote>
<p>Do you have a lot of JavaScript projects using GitHub repositories? How are you managing dependencies among those projects? Developers might be struggling to manage a large set of GitHub repositories and Business might be spending a lot of money to manage them. The potential solution to this problem is Monorepo. You can put all of your JavaScript projects regardless of framework (angular, vue.js, react.js) in a single GitHub repository called monorepo. This will solve the problem and you will not have many Git projects. However, nothing is free! So there are many challenges with monorepo that developers are facing. I will explain all of the challenges and show you how you can solve them. Also I will create monorepo from scratch in this article using angular projects as samples.</p>
</blockquote>
<h2 id="what-is-monorepo">What is Monorepo?</h2>
<p>Single Git Repository holds source code of multiple applications and libraries.</p>
<h2 id="why-to-use-monorepo">Why to use Monorepo?</h2>
<p>Monorepo in organization helps you to get below benefits:</p>
<ul>
<li><strong>Shared Code</strong>: Reuse code validation, tooling code, and libraries code in applications in a single repository.</li>
<li><strong>Atomic Changes</strong>: Change the button component in the shared library and update the application that uses that button in the same commit.</li>
<li><strong>Developer Mobility</strong>: Developers can easily contribute to any applications or libraries by just going to one source code repository.</li>
</ul>
<h2 id="what-should-i-know-before-using-monorepo">What should I know before using Monorepo?</h2>
<p>If you have already developed couple of JavaScript projects then you are good to go and learn Monorepo. For this article if you have basic knowledge of Angular then it will be good since I will create angular projects in example monorepo.</p>
<h2 id="where-is-source-code-for-nx-monorepo">Where is source code for Nx Monorepo?</h2>
<p>Here is the <a href="https://www.npmjs.com/package/nx">Nx npm package</a> that you can use from npm. For this article I will demonstrate code that is hosted in github at below location:</p>
<ul>
<li><a href="https://github.com/rupeshtiwari/coding-examples-angular-monorepo-with-nx">Angular Monorepo Nrwl.NX sample</a></li>
<li><a href="https://github.com/rupeshtiwari/coding-examples-angular-monorepo-nx-azure-ci-cd">Angular Monorepo Nrwl.NX sample with advance Microsoft Azure CI/CD Pipeline</a></li>
</ul>
<h2 id="challenges-with-monorepo">Challenges with Monorepo?</h2>
<p>Challenges are how to identify which application depends on how many projects? How to constrain dependency so that it reduces cyclic dependency issues and organizes code structure? How to only compile the project which has only changed? How to only run tests for the changed projects? How to increase tooling speed for a large set of projects in a single monorepo? How to manage versions? All of these are very important requirements. Nx.Nrwl monorepo framework is an excellent framework to manage any number of JavaScript projects in just one GitHub Repo. With Nx.Nrwl Monorepo you get facility to visualize which project depends on others. All of the challenges that I mentioned could be resolved by Nx Monorepo. Stay tuned and read this article till the end to create your angular monorepo from scratch using the Nx Monorepo framework.</p>
<h2 id="how-to-manage-versions-and-parallel-development">How to manage versions and parallel development?</h2>
<p>You could have your own policy as per your organization’s need to manage parallel development, release, fix, development, merging, major release, maintenance related work. However, I am suggesting one option. Let’s see if that makes sense to you.</p>
<h3 id="managing-releases-with-monorepo">Managing Releases with Monorepo</h3>
<p>Create a branch for release number. Example <code class="language-plaintext highlighter-rouge">releases/11.0</code> treat this branch as active working branch. You can treat this as your <code class="language-plaintext highlighter-rouge">dev</code> branch.</p>
<p><img src="https://i.imgur.com/SQUdxd4.png" alt="" class="full" /></p>
<h3 id="working-on-feature-branch-with-monorepo">Working on Feature Branch with Monorepo</h3>
<p>For adding new feature or working on some new PBI or usecase. Create a features folder and put PBI number as new branch in features folder work on development and merge them in to release branch.
Example: <code class="language-plaintext highlighter-rouge">features/PBI123/appendOrder</code></p>
<h3 id="managing-major-releases-with-monorepo">Managing Major Releases with Monorepo</h3>
<p>In order to do major release create a new branch <code class="language-plaintext highlighter-rouge">11.1</code> as major release branch. And if you want to apply fix on <code class="language-plaintext highlighter-rouge">11.1</code> then create another branch <code class="language-plaintext highlighter-rouge">11.1.1</code> work on fixes and merge them back to release branch <code class="language-plaintext highlighter-rouge">11.0</code> for future release.</p>
<p><img src="https://i.imgur.com/F4rqVdP.png" alt="" class="full" /></p>
<h2 id="what-developer-will-learn-in-this-article">What Developer will learn in this article?</h2>
<p>In this article, I will walk you through the steps to create one <a href="https://nx.dev/">nx monorepo</a> workspace with Angular application and libraries from empty preset.</p>
<ol>
<li>We will use Azure Pipeline Caching to improve build speed by caching npm packages.</li>
<li>We will publish our application to npm from pipeline in automated fashion.</li>
</ol>
<h3 id="in-the-monorepo-we-will-do-below-work">In the Monorepo we will do below work</h3>
<ol>
<li>I need <strong>Karma</strong> as test runner</li>
<li>I <strong>don’t need e2e test</strong> for applications</li>
<li>I want my <strong>applications to be publishable</strong></li>
<li>I want my <strong>libraries to be not publishable</strong></li>
<li>Create <strong>Azure CI/CD pipelines</strong></li>
<li>Deploy <strong>Application to NPM registry</strong></li>
</ol>
<h2 id="development-environment-setup">Development Environment Setup</h2>
<p>Hey developer, make sure you have <code class="language-plaintext highlighter-rouge">node.js</code> installed which is greater than <code class="language-plaintext highlighter-rouge">V10</code>. I have installed <code class="language-plaintext highlighter-rouge">node.js v14</code>.</p>
<h3 id="visual-studio-code-vscode">Visual Studio Code (VsCode)</h3>
<p>I am using Vs Code and installed nx console extension to get inbuilt way of creating apps/libs from nx console only.</p>
<h3 id="visual-studio-code-extensions">Visual Studio Code Extensions</h3>
<p>Install below VsCode extensions for better development experience:</p>
<ul>
<li><a href="https://nx.dev/latest/angular/getting-started/console">Nx Console</a> in your vs code to create components without writing code.
<img src="https://i.imgur.com/gWKa76u.png =250x200" alt="" /></li>
<li><a href="https://marketplace.visualstudio.com/items?itemName=johnpapa.angular-essentials">Angular Essentials Extensions by John Papa</a>
<img src="https://i.imgur.com/bUO042c.png" alt="" class="full" /></li>
<li><a href="https://marketplace.visualstudio.com/items?itemName=PKief.material-icon-theme">Material Theme VS Code Extension</a>: In order to see the icons for each file you must update the vscode settings. Take the custom Vs Code Settings of VsCode from <a href="#vscode-settings">this article</a>.</li>
<li>
<p><a href="https://marketplace.visualstudio.com/items?itemName=ms-azure-devops.azure-pipelines">Azure Pipelines</a> for validations of yml file
<img src="https://i.imgur.com/kZIWqaY.png" alt="" class="full" /></p>
</li>
<li>Install <a href="https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml">yaml visual studio extension</a>
<img src="https://i.imgur.com/Sqsa8gx.png" alt="" class="full" /></li>
</ul>
<p><strong>Below are optional but recommended extensions</strong></p>
<ul>
<li><a href="https://marketplace.visualstudio.com/items?itemName=wayou.vscode-todo-highlight">TODO Highlighter</a> : Optional it helps to highlight todo and fixme comments.
<img src="https://i.imgur.com/JT28IhH.png" alt="" class="full" /></li>
</ul>
<p><strong>FixMe Comments</strong>
FIXME: Comments are some things that you are planning to fix within the current sprint.<br />
Example: Fixing Api Service to call Server Web API within this sprint only.
Stories having FIXME are alarming for you. Make sure all FIXME’s are addressed before marking stories are done. 📓 Notice: Once you fix the code then remove the comment 😄
<img src="https://i.imgur.com/Zgbdf9N.png" alt="" class="full" /></p>
<p><strong>Todo Comments</strong>
TODO: comments are something that you are planning to work on future sprints.
Example: Working on MCQ API Service to call server side API that has to be done on future sprint.
<img src="https://i.imgur.com/i48KY0n.png" alt="" class="full" /></p>
<p>Check all todo comments <code class="language-plaintext highlighter-rouge">CTRL + SHIFT + P</code>
<img src="https://i.imgur.com/KUq3DFR.png" alt="" class="full" /></p>
<ul>
<li><a href="https://marketplace.visualstudio.com/items?itemName=KnisterPeter.vscode-commitizen">Conventional Commit Comments</a>
<img src="https://i.imgur.com/uDw6zDP.png" alt="" class="full" />
<a href="https://gist.github.com/qoomon/5dfcdf8eec66a051ecd85625518cfd13#examples">Check out some examples of conentional commit comments</a></li>
</ul>
<h3 id="vscode-settings">Vscode Settings</h3>
<p>If you want to get the custom icons for your files then make sure you have installed VSCode Materials and copy below settings and paste in your local VSCode settings.json file located at <code class="language-plaintext highlighter-rouge">%appdata%\Code\User\settings.json</code></p>
<script src="https://gist.github.com/a4190a29aac0132082430cdb8a2e85ad.js"> </script>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="nl">"material-icon-theme.folders.associations"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"itops"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Vm"</span><span class="p">,</span><span class="w">
</span><span class="nl">"presentation"</span><span class="p">:</span><span class="s2">"Views"</span><span class="p">,</span><span class="w">
</span><span class="nl">"aggregate-score"</span><span class="p">:</span><span class="s2">"Sublime"</span><span class="p">,</span><span class="w">
</span><span class="nl">"aspects"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apollo"</span><span class="p">,</span><span class="w">
</span><span class="nl">"contracts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Scala"</span><span class="p">,</span><span class="w">
</span><span class="nl">"message-handlers"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Serverless"</span><span class="p">,</span><span class="w">
</span><span class="nl">"base"</span><span class="p">:</span><span class="w"> </span><span class="s2">"core"</span><span class="p">,</span><span class="w">
</span><span class="nl">"branding"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Shared"</span><span class="p">,</span><span class="w">
</span><span class="nl">"main"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Stack"</span><span class="p">,</span><span class="w">
</span><span class="nl">"composition"</span><span class="p">:</span><span class="w"> </span><span class="s2">"include"</span><span class="p">,</span><span class="w">
</span><span class="nl">"modules"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Cluster"</span><span class="p">,</span><span class="w">
</span><span class="nl">"partials"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Mock"</span><span class="p">,</span><span class="w">
</span><span class="nl">"filters"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Terraform"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sharedviews"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Svelte"</span><span class="p">,</span><span class="w">
</span><span class="nl">"dashboard"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Gulp"</span><span class="p">,</span><span class="w">
</span><span class="nl">"ui-core"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Review"</span><span class="p">,</span><span class="w">
</span><span class="nl">"dev"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Intellij"</span><span class="p">,</span><span class="w">
</span><span class="nl">"ngmaterial"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Theme"</span><span class="p">,</span><span class="w">
</span><span class="nl">"ckeditor5"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Mobile"</span><span class="p">,</span><span class="w">
</span><span class="nl">"launch"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Docker"</span><span class="p">,</span><span class="w">
</span><span class="nl">"editor"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Custom"</span><span class="p">,</span><span class="w">
</span><span class="nl">"toolbar"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Tools"</span><span class="p">,</span><span class="w">
</span><span class="nl">"pubsub"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Event"</span><span class="p">,</span><span class="w">
</span><span class="nl">"message-handler"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Delta"</span><span class="p">,</span><span class="w">
</span><span class="nl">"api-composition"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Mock"</span><span class="p">,</span><span class="w">
</span><span class="nl">"dto"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Gradle"</span><span class="p">,</span><span class="w">
</span><span class="nl">"interfaces"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Animation"</span><span class="p">,</span><span class="w">
</span><span class="nl">"global-error-handler"</span><span class="p">:</span><span class="s2">"Error"</span><span class="p">,</span><span class="w">
</span><span class="nl">"root"</span><span class="p">:</span><span class="s2">"Svg"</span><span class="w">
</span><span class="p">}</span><span class="err">,</span><span class="w">
</span><span class="nl">"material-icon-theme.files.associations"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"*.state.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Silverstripe"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.dto.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Gradle"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.module.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Racket"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.routing.module.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"R"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.store.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Storybook"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.tests.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Test-ts"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.controller.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Angular-component"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.enum.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Jinja"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.service.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Quasar"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.model.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Shaderlab"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.viewmodel.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Pug"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.command.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Email"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.event.ts"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Stencil"</span><span class="p">,</span><span class="w">
</span><span class="nl">"*.ts"</span><span class="p">:</span><span class="s2">"Idris"</span><span class="w">
</span><span class="p">}</span><span class="err">,</span><span class="w">
</span></code></pre></div></div>
<p>You will get below icons once updating settings file.</p>
<p><img src="https://i.imgur.com/P2D9gy6.png" alt="" class="full" /></p>
<h2 id="enable-karma-test-runner-in-nx-monorepo">Enable Karma Test Runner in Nx Monorepo</h2>
<p>When you create nx monorepo with <code class="language-plaintext highlighter-rouge">Angular</code> or <code class="language-plaintext highlighter-rouge">React</code> or <code class="language-plaintext highlighter-rouge">Next.js</code> or <code class="language-plaintext highlighter-rouge">Gatsby</code> or <code class="language-plaintext highlighter-rouge">next.js</code> or <code class="language-plaintext highlighter-rouge">Web Components</code> then it will give you <code class="language-plaintext highlighter-rouge">JEST</code> as unit test runner. In case you want to get <code class="language-plaintext highlighter-rouge">karma</code> test runner then you have to create Empty workspace and then add <code class="language-plaintext highlighter-rouge">@nrwl/angular</code> then add new application or lib with karma unit test runner.</p>
<h2 id="creating-first-nx-monorepo-using-cli">Creating First Nx Monorepo using CLI</h2>
<p>In order to create your first Nx Monorepo workspace run below script:</p>
<p><code class="language-plaintext highlighter-rouge">npx create-nx-workspace@latest</code></p>
<ul>
<li>Give workspace name: <code class="language-plaintext highlighter-rouge">myorg</code></li>
<li>Select Layout: I select Angular Application (A workspace with a single Angular Application)</li>
<li>Application Name: cutepuppies-admin</li>
<li>Stylesheet Format: SASS</li>
<li>Linter: TSLINT</li>
<li>Nx Cloud: No ( this is default )</li>
</ul>
<p><img src="https://i.imgur.com/iMK98Jr.png" alt="" class="full" /></p>
<p>It will create the workspace with angular app that we defined.</p>
<p><img src="https://i.imgur.com/X8fvsUc.png" alt="" class="full" /></p>
<p>When you will use their template they will add end to end test for application. They will also add jest test runner.</p>
<p><img src="https://i.imgur.com/GLOkklz.png" alt="" class="full" /></p>
<p>In my workspace, I do not want Jest and I do not want e2e tests. So I am creating my workspace with empty preset next.</p>
<h2 id="creating-nx-monorepo-with-custom-settings">Creating Nx Monorepo with Custom Settings</h2>
<ol>
<li>
<p>Run <code class="language-plaintext highlighter-rouge">npx create-nx-workspace@latest</code></p>
</li>
<li>
<p>Give organization name: <code class="language-plaintext highlighter-rouge">curise</code></p>
</li>
<li>
<p>Select angular application for the workspace. <code class="language-plaintext highlighter-rouge">angular</code></p>
</li>
</ol>
<p>This will install <code class="language-plaintext highlighter-rouge">@nrwl/angular</code> plugin to give you tools to create angular lib and apps.</p>
<p><img src="https://i.imgur.com/zia6hBU.png" alt="" class="full" /></p>
<ol>
<li>Then enter angular application name: admin</li>
</ol>
<p><img src="https://i.imgur.com/jn9PH3x.png" alt="" class="full" /></p>
<ol>
<li>Next select Style format as SASS</li>
<li>Default Linter: Choose TSLINT since I have angular app</li>
<li>Nx Cloud give caching over cloud: No ( it needs paid subscription plan)</li>
</ol>
<p><img src="https://i.imgur.com/8XevXrP.png" alt="" class="full" /></p>
<p>Run the script:
<img src="https://i.imgur.com/roU0Y90.png" alt="" class="full" /></p>
<h2 id="understanding-nx-monorepo-workspace">Understanding Nx Monorepo Workspace</h2>
<p><img src="https://i.imgur.com/bNRYcQI.png" alt="" class="full" /></p>
<ol>
<li><strong>Apps</strong></li>
<li><strong>libs</strong></li>
<li><strong>tools</strong>
Tools folder has tsconfig file that means u can write typescript files and compile them in JS.</li>
<li><strong>decorate-angular-cli.js</strong></li>
<li><strong>jest.preset.js</strong></li>
<li><strong>angular.json</strong>
It create angular.json file to manage your workspace. However, if you want to create empty monorepo project then it creates <strong>workspace.json</strong> file.</li>
</ol>
<h2 id="creating-projects-in-nx-monorepo">Creating Projects in Nx Monorepo</h2>
<p>Run <code class="language-plaintext highlighter-rouge">nx build</code></p>
<p><img src="https://i.imgur.com/Pig0oei.png" alt="" class="full" /></p>
<p>Showing <code class="language-plaintext highlighter-rouge">dist</code> folder.</p>
<p><img src="https://i.imgur.com/wIjX2a2.png" alt="" class="full" /></p>
<h2 id="empty-nx-monorepo-advantage-and-disadvantages">Empty Nx Monorepo Advantage and Disadvantages</h2>
<p>After creating empty workspace.</p>
<p><strong>Benefits of Empty Workspace</strong></p>
<ol>
<li>You can use <code class="language-plaintext highlighter-rouge">karma</code> unit test runner</li>
<li>You can use <code class="language-plaintext highlighter-rouge">workspace.json</code> to configure any apps and libs.</li>
</ol>
<p><strong>Disadvantages of empty workspace</strong>:</p>
<ol>
<li>You have to write the angular cli decorator for <code class="language-plaintext highlighter-rouge">Nx CLI</code></li>
</ol>
<p>Regardless of empty or pre-populated workspace you must create your own <code class="language-plaintext highlighter-rouge">CI/CD</code> Pipelines and Nx commands for your pipelines.</p>
<h2 id="creating-new-empty-nx-monorepo-workspace">Creating new Empty Nx Monorepo workspace</h2>
<p>Now we will create our monorepo workslpace that we will continue using it. We will start from scratch empty monorepo.</p>
<p>Creating empty Nx Monorepo workspace is our final option for this example. So let’s create desired working folder and from that location open <code class="language-plaintext highlighter-rouge">powershell</code> on windows machine I found working on <code class="language-plaintext highlighter-rouge">git bash commands</code> were not working nicely.</p>
<p>Run below script to create empty workspace</p>
<p><code class="language-plaintext highlighter-rouge">npx create-nx-workspace@latest --preset=empty</code></p>
<p>Above script will ask you</p>
<ol>
<li>organization name and</li>
<li>Want to use Nx Cloud or not.</li>
</ol>
<p><img src="https://i.imgur.com/2nPySWd.png" alt="" class="full" /></p>
<p>Notice it created the workspace</p>
<p><img src="https://i.imgur.com/ApKE767.png" alt="" class="full" /></p>
<p>So Now I get empty app and library folders.</p>
<p><img src="https://i.imgur.com/vl5KtTa.png" alt="" class="full" /></p>
<p>This is my dependencies that I get in package.json</p>
<p><img src="https://i.imgur.com/Y4qQNIe.png" alt="" class="full" /></p>
<h2 id="what-nx-cli-does">What NX CLI does?</h2>
<p><code class="language-plaintext highlighter-rouge">Nx CLI</code> commands will invoke <code class="language-plaintext highlighter-rouge">Angular CLI</code>. The <code class="language-plaintext highlighter-rouge">Nx CLI</code> simply does some optimizations before invoking the <code class="language-plaintext highlighter-rouge">Angular CLI</code>. <code class="language-plaintext highlighter-rouge">Nx Cli</code> can execute all of the <code class="language-plaintext highlighter-rouge">ng cli</code> commands. The benefit you get with <code class="language-plaintext highlighter-rouge">Nx CLI</code> is <strong>Computation Caching</strong> which gives faster execution of tasks. Also Nx CLI is technology agnostic You can use <code class="language-plaintext highlighter-rouge">Nx CLI</code> to run <code class="language-plaintext highlighter-rouge">angular</code>, <code class="language-plaintext highlighter-rouge">react</code>, <code class="language-plaintext highlighter-rouge">JavaScript</code> or any project scripts.
So it gives uniformity in your team to always run <code class="language-plaintext highlighter-rouge">Nx CLI</code> in your workspace regardless of any front-end technology you are using..</p>
<h2 id="using-nx-cli-over-angular-cli">Using Nx CLI over Angular CLI</h2>
<p>In order to point <code class="language-plaintext highlighter-rouge">ng cli</code> to <code class="language-plaintext highlighter-rouge">Nx CLI</code> we have to run <strong>decorate-angular-cli.js</strong> file. Because of <code class="language-plaintext highlighter-rouge">symlinking</code> you can still type <code class="language-plaintext highlighter-rouge">ng build</code> or <code class="language-plaintext highlighter-rouge">ng test</code> or <code class="language-plaintext highlighter-rouge">ng lint</code> in the terminal. The ng command, in this case, will point to nx, which will perform optimizations before invoking <code class="language-plaintext highlighter-rouge">ng</code>.</p>
<p>After install we can execute this decorator file.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nl">"postinstall"</span><span class="p">:</span><span class="w"> </span><span class="s2">"node ./decorate-angular-cli.js && ngcc --properties es2015 browser module main --first-only --create-ivy-entry-points"</span><span class="err">,</span><span class="w">
</span></code></pre></div></div>
<h2 id="how-to-opt-out-of-nx-cli">How to opt out of NX CLI</h2>
<p>Although I don’t recommend this option :-1:<br />
However, in case you want to opt out of <code class="language-plaintext highlighter-rouge">NX CLI</code> then follow below steps:</p>
<ol>
<li>Replace occurrences of nx with ng in your package.json</li>
<li>Remove the script from your postinstall script in your package.json</li>
<li>Delete and reinstall your node_modules</li>
</ol>
<p>We are not doing this though.</p>
<h2 id="understanding-nx-workspace-scripts">Understanding NX Workspace Scripts</h2>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"nx"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx"</span><span class="p">,</span><span class="w">
</span><span class="nl">"start"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx serve"</span><span class="p">,</span><span class="w">
</span><span class="nl">"build"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx build"</span><span class="p">,</span><span class="w">
</span><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx test"</span><span class="p">,</span><span class="w">
</span><span class="nl">"lint"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx workspace-lint && nx lint"</span><span class="p">,</span><span class="w">
</span><span class="nl">"e2e"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx e2e"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:apps"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:apps"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:libs"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:libs"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:build"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:build"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:e2e"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:e2e"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:test"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:test"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:lint"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:lint"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected:dep-graph"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected:dep-graph"</span><span class="p">,</span><span class="w">
</span><span class="nl">"affected"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx affected"</span><span class="p">,</span><span class="w">
</span><span class="nl">"format"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx format:write"</span><span class="p">,</span><span class="w">
</span><span class="nl">"format:write"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx format:write"</span><span class="p">,</span><span class="w">
</span><span class="nl">"format:check"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx format:check"</span><span class="p">,</span><span class="w">
</span><span class="nl">"update"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx migrate latest"</span><span class="p">,</span><span class="w">
</span><span class="nl">"workspace-generator"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx workspace-generator"</span><span class="p">,</span><span class="w">
</span><span class="nl">"dep-graph"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx dep-graph"</span><span class="p">,</span><span class="w">
</span><span class="nl">"help"</span><span class="p">:</span><span class="w"> </span><span class="s2">"nx help"</span><span class="w">
</span><span class="p">}</span><span class="err">,</span><span class="w">
</span></code></pre></div></div>
<h2 id="creating-angular-library-using-nx-console">Creating Angular Library using Nx Console</h2>
<p>I will use <code class="language-plaintext highlighter-rouge">nx console</code> to create application.</p>
<p>Since I want to create Angular applications and libraries I need <code class="language-plaintext highlighter-rouge">@nrwl/angular</code> node package to be
installed.</p>
<p>please run below script to install <code class="language-plaintext highlighter-rouge">@nrwl/angular</code></p>
<p><code class="language-plaintext highlighter-rouge">npm i -D @nrwl/angular</code></p>
<p><img src="https://i.imgur.com/uq8rqan.png" alt="" class="full" /></p>
<p>Select Nx, then select generate and then select Angular Library.</p>
<p><img src="https://i.imgur.com/NzsCaoS.png" alt="" class="full" /></p>
<p>Fill out the required fields. I want to create logger angular library under lib/branding folder with karma testing framework.</p>
<p><img src="https://i.imgur.com/rNAerhj.png" alt="" class="full" /></p>
<p>This is my dry run result</p>
<p><img src="https://i.imgur.com/Erh9MXZ.png" alt="" class="full" /></p>
<p>Logger project is created successfully.
<img src="https://i.imgur.com/OM0IQE3.png" alt="" class="full" /></p>
<h2 id="how-to-stop-dryrun-while-using-nx-console">How to stop dryRun while using Nx Console?</h2>
<p>Currently there is no way you can stop dry run therefore, use the script to create application or lib.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nx generate @nrwl/angular:library <span class="nt">--name</span><span class="o">=</span>utils <span class="nt">--directory</span><span class="o">=</span>branding <span class="nt">--importPath</span><span class="o">=</span>@myor/branding-utils <span class="nt">--prefix</span><span class="o">=</span>myorg-branding-utils <span class="nt">--tags</span><span class="o">=</span>scope:branding,type:branding-utils <span class="nt">--no-interactive</span> <span class="nt">--dry-run</span>
</code></pre></div></div>
<h2 id="creating-js-project-using-nx-console">Creating JS project using NX Console</h2>
<p>NX Monorepo only support <code class="language-plaintext highlighter-rouge">JEST</code> or <code class="language-plaintext highlighter-rouge">NONE</code> unit test runner for Javascript project.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nx generate @nrwl/workspace:library <span class="nt">--name</span><span class="o">=</span>ckeditor5 <span class="nt">--directory</span><span class="o">=</span>branding <span class="nt">--importPath</span><span class="o">=</span>@myorg/branding-ckeditor5 <span class="nt">--prefix</span><span class="o">=</span>myorg-branding-ckeditor5 <span class="nt">--tags</span><span class="o">=</span>scope:branding,type:branding-ckeditor5 <span class="nt">--no-interactive</span> <span class="nt">--js</span> <span class="nt">--unitTestRunner</span><span class="o">=</span>none <span class="nt">--dry-run</span>
</code></pre></div></div>
<h2 id="how-to-create-run-command-in-nx-monorepo">How to create Run Command in NX Monorepo</h2>
<p>Nx Run Command helps to create custom commands</p>
<p><img src="https://i.imgur.com/6OPfuS6.png" alt="" class="full" /></p>
<p><strong>Target Name</strong> : Bane of the target like <code class="language-plaintext highlighter-rouge">build</code>
<strong>Project Name</strong> : is the name of the project in the workspace.json file : <code class="language-plaintext highlighter-rouge">branding-ckeditor</code></p>
<p><strong>Command</strong>: <code class="language-plaintext highlighter-rouge">npm run build</code></p>
<p><strong>cwd</strong> location of the src folder of the project</p>
<p><strong>outputs</strong> where the build artifacts will be stored.
<img src="https://i.imgur.com/BS0ulgd.png" alt="" class="full" /></p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nx generate @nrwl/workspace:run-commands <span class="nt">--name</span><span class="o">=</span>build <span class="nt">--command</span><span class="o">=</span><span class="s1">'npm run build'</span> <span class="nt">--project</span><span class="o">=</span>branding-ckeditor5 <span class="nt">--cwd</span><span class="o">=</span>libs/branding/ckeditor5/src <span class="nt">--outputs</span><span class="o">=</span>libs/branding/ckeditor5 <span class="nt">--no-interactive</span> <span class="nt">--dry-run</span>
</code></pre></div></div>
<p>This will update the <code class="language-plaintext highlighter-rouge">workspace.json</code> and <code class="language-plaintext highlighter-rouge">nx.json</code></p>
<p><img src="https://i.imgur.com/JYPutPu.png" alt="" class="full" /></p>
<p>Now you can run <code class="language-plaintext highlighter-rouge">nx build branding-ckeditor5</code></p>
<h2 id="step-create-angular-app-using-nx-console">Step: Create Angular App using Nx Console</h2>
<p>Let’s select an angular application this time to create cutepuppies-admin apps.
<img src="https://i.imgur.com/XRG6STh.png" alt="" class="full" /></p>
<p>Name: cutepuppies-admin
<img src="https://i.imgur.com/QNPMMpp.png" alt="" class="full" /></p>
<p>e2eTestRunner: none ( I don’t want this test )
<img src="https://i.imgur.com/FzeuvBR.png" alt="" class="full" /></p>
<p>Routing: Generates a routing module select this.</p>
<p><img src="https://i.imgur.com/oW0DuLZ.png" alt="" class="full" /></p>
<p>Tags: scope:itops,type:itops-cutepuppies-admin
Unit Test Runner: Karma
<img src="https://i.imgur.com/x75qAxN.png" alt="" class="full" /></p>
<p>Click on the Run button and finally select SASS while it is asking for which style format.
See my app is created
<img src="https://i.imgur.com/fAz1Q1y.png" alt="" class="full" /></p>
<h2 id="adding-nx-cli-decorator-for-nx-empty-workspace">Adding Nx CLI Decorator for Nx Empty Workspace</h2>
<h2 id="step--lets-create-logging-service--tests">Step : Let’s Create Logging Service & Tests</h2>
<p>Let’s create a loggingService inside our logger project.
Use Nx Console to create angular service.
<img src="https://i.imgur.com/ynFGEdp.png" alt="" class="full" /></p>
<p>This is dry run result of angular service creation
<img src="https://i.imgur.com/oyCwe1b.png" alt="" class="full" /></p>
<p>Fill out the form add below information
<img src="https://i.imgur.com/SZDx1N5.png" alt="" class="full" /></p>
<p>Name: logging
project: branding-logger
Notice Logger service created
<img src="https://i.imgur.com/7aLT4Yn.png" alt="" class="full" /></p>
<p>Lets run the test and see if the test is passing or not. Since I am using main branch instead of master branch I will update my scripts to take base as main branch only.</p>
<p><img src="https://i.imgur.com/LQmjidP.png" alt="" class="full" /></p>
<p>Since only logging service is added if I run the test only one project test will run.
Run npm run affected:test
<img src="https://i.imgur.com/owTmYZl.png" alt="" class="full" /></p>
<h2 id="step--lets-take-logger-dependency-on-our-application">Step : Let’s take logger dependency on our application</h2>
<p>Next lest run the build
<img src="https://i.imgur.com/2OAeDSS.png" alt="" class="full" /></p>
<p>npm run affected:build
<img src="https://i.imgur.com/wfr90IZ.png" alt="" class="full" /></p>
<p>Notice the app is building and the dist folder is created with one app.</p>
<p><img src="https://i.imgur.com/noosk8r.png" alt="" class="full" /></p>
<p>Now let’s use a logging service to log something and run our application next.</p>
<p>Go to libs/branding/logger/src/index.ts and export the logger service.</p>
<p><img src="https://i.imgur.com/7w7SM23.png" alt="" class="full" /></p>
<p>Next go to apps\cutepuppies-admin\src\app\app.component.ts and add logging service and log in constructor.
<img src="https://i.imgur.com/QGpa6da.png" alt="" class="full" /></p>
<p>Lets serve our application run below script</p>
<p>nx serve cutepuppies-admin</p>
<p><img src="https://i.imgur.com/wyde0hs.png" alt="" class="full" /></p>
<p>Navigate to http://localhost:4200 and see in console log hello world is coming.</p>
<p><img src="https://i.imgur.com/FMj0LWO.png" alt="" class="full" /></p>
<p>Now lets build the application again and go to the dist folder and run below script from dist folder</p>
<p>cd dist/apps/cutepuppies-admin && npx http-server -o</p>
<p>Notice our app is running and I see the log as well.</p>
<p><img src="https://i.imgur.com/GveyEna.png" alt="" class="full" /></p>
<p>Creating Cute Puppies Client
<img src="https://i.imgur.com/KiVPRi2.png" alt="" class="full" /></p>
<p>Created Sales/PuppyEditor
<img src="https://i.imgur.com/LrT06gn.png" alt="" class="full" /></p>
<p>Creating sales/puppies</p>
<p><img src="https://i.imgur.com/oRMOvDt.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/M4nNi9M.png" alt="" class="full" /></p>
<h2 id="adding-users-library-under-customers-service">Adding Users Library under Customers Service</h2>
<p><img src="https://i.imgur.com/fs31irh.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/KbjCP4n.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/69WJwUz.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/fJda329.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/0EXHyZr.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/xsETtU5.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/fZv6tcS.png" alt="" class="full" /></p>
<p>Adding sales puppies route to client app.
<img src="https://i.imgur.com/B5PJ3BX.png" alt="" class="full" /></p>
<p>Adding puppy editor route in admin app</p>
<p><img src="https://i.imgur.com/XPNXHTP.png" alt="" class="full" /></p>
<h2 id="creating-add-puppy-component-on-puppy-editor-project">Creating Add-Puppy Component on Puppy Editor project</h2>
<p><img src="https://i.imgur.com/KQcjsJL.png" alt="" class="full" /></p>
<p>Declaring Module: libs/sales/puppy-editor/src/lib/sales-puppy-editor.module.ts</p>
<p><img src="https://i.imgur.com/4Kn3o3C.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/SJO5aT3.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/Dc4DbjL.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/u6Mj2Jx.png" alt="" class="full" /></p>
<p>Check Dry Run result and confirm
<img src="https://i.imgur.com/09Ly6gw.png" alt="" class="full" /></p>
<p>and select Run
<img src="https://i.imgur.com/BVo8q7V.png" alt="" class="full" /></p>
<p>Component Created</p>
<p><img src="https://i.imgur.com/3fGPE1t.png" alt="" class="full" /></p>
<p>Component is good
<img src="https://i.imgur.com/Aze7pmz.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/hzcmhnM.png" alt="" class="full" /></p>
<h2 id="add-puppies-component-in-the-customerspuppies-using-nx-console">Add Puppies component in the Customers/Puppies using Nx Console.</h2>
<p><img src="https://i.imgur.com/I6UJayT.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/ozBq2Tg.png" alt="" class="full" /></p>
<p>To get the path of the component copy the relative path of lib folder of sales/puppies <code class="language-plaintext highlighter-rouge">/libs/sales/puppies/src/lib</code></p>
<p><img src="https://i.imgur.com/6Wh60db.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/ybNcmwX.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/1RdC1PS.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/VuJK34K.png" alt="" class="full" /></p>
<p>Select Run to create component</p>
<p><img src="https://i.imgur.com/T57LNcC.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/mFVIQJI.png" alt="" class="full" />
<img src="https://i.imgur.com/nMdYTc8.png" alt="" class="full" />
Lets update the routing for puppies
<img src="https://i.imgur.com/zfWzwRy.png" alt="" class="full" />
Add route for add puppy in puppy editor
<img src="https://i.imgur.com/s6dEpx8.png" alt="" class="full" /></p>
<h3 id="adding-routes-for-admin-app">Adding routes for Admin App</h3>
<p>Add route in admin app: add below route to navigate to puppy editor default component
<img src="https://i.imgur.com/CEJQgz9.png" alt="" class="full" /></p>
<h3 id="adding-routes-for-client-app">Adding routes for Client App</h3>
<p>Add route in client app: add below route to navigate to puppies module’s default component.
<img src="https://i.imgur.com/eLa5JoV.png" alt="" class="full" /></p>
<h3 id="serving-admin-component">Serving Admin component</h3>
<p>Lets add scripts to serve both admin and client app</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nl">"start-admin"</span><span class="p">:</span><span class="s2">"nx serve cutepuppies-admin --port=4210 -o"</span><span class="err">,</span><span class="w">
</span><span class="nl">"start-client"</span><span class="p">:</span><span class="s2">"nx serve cutepuppies-client --port=4211 -o"</span><span class="err">,</span><span class="w">
</span></code></pre></div></div>
<p><img src="https://i.imgur.com/nEnnhWf.png" alt="" class="full" /></p>
<p>Lets clean app component HTML</p>
<p><img src="https://i.imgur.com/wmIXSdj.png" alt="" class="full" /></p>
<p>Lets run the admin app <code class="language-plaintext highlighter-rouge">npm run start-admin</code></p>
<p><img src="https://i.imgur.com/Kn9feLH.png" alt="" class="full" /></p>
<p>Admin Site is up and running
http://localhost:4210/puppies-editor</p>
<p><img src="https://i.imgur.com/VE8KYyh.png" alt="" class="full" /></p>
<h2 id="serving-client-app">Serving client app</h2>
<p>app component
<img src="https://i.imgur.com/cAGJAjc.png" alt="" class="full" /></p>
<p>Run <code class="language-plaintext highlighter-rouge">npm run start-client</code>
<img src="https://i.imgur.com/SW5kU3k.png" alt="" class="full" /></p>
<p>Client site is running : http://localhost:4211/puppies
<img src="https://i.imgur.com/o0iSvPJ.png" alt="" class="full" /></p>
<h2 id="monorepo-workspace-architecture">MonoRepo Workspace Architecture</h2>
<p><strong>Apps</strong></p>
<ul>
<li>Apps are IT-Ops systems</li>
<li>It can depend on anything</li>
<li>Only apps are buildable and publishable to npm repositories.</li>
</ul>
<p><strong>Libs</strong></p>
<ul>
<li>Libs are services</li>
<li>Services should have multiple projects</li>
<li>Services should not depend on each other</li>
<li>Services projects should not depend on each other. Unless it is configured to do so.</li>
<li>Services projects can depend on Branding</li>
<li>Services projects are not buildable and not publishable</li>
<li>Services projects has a libs folder which is equivalent to modules folder.</li>
</ul>
<h2 id="monorepo-workspace-dependency-graph">MonoRepo Workspace Dependency Graph</h2>
<p>Run <code class="language-plaintext highlighter-rouge">nx dep-graph --base=main</code></p>
<p><img src="https://i.imgur.com/1bA7G5Z.png" alt="" class="full" /></p>
<h2 id="creating-build-ci-pipeline">Creating Build CI pipeline</h2>
<p>Install <a href="https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml">yaml visual studio extension</a>
<img src="https://i.imgur.com/T9rDvmQ.png" alt="" class="full" /></p>
<p>Install <a href="https://marketplace.visualstudio.com/items?itemName=ms-azure-devops.azure-pipelines">Azure Pipelines</a> for validations of yml file</p>
<p><img src="https://i.imgur.com/D9VxmG9.png" alt="" class="full" /></p>
<h3 id="caching-azure-pipeline">Caching Azure Pipeline</h3>
<p>We will install node modules and cache it so that for future build it will read from the cache location. If package-lock.json is changed it will refresh the cache.</p>
<p>We are going to use <strong>Restores and saves pipeline artifacts with Universal Packages</strong> to cache the ci pipelines. So Install the Task in your azure devops from marketplace.</p>
<p><a href="https://marketplace.visualstudio.com/items?itemName=1ESLighthouseEng.PipelineArtifactCaching">Restores and saves pipeline artifacts with Universal Packages</a>
<img src="https://i.imgur.com/pNN6xl4.png" alt="" class="full" /></p>
<p>Since we are use pipeline cache, we need to create Artifacts Feed in Azure devops to store the node_modules folders. <a href="https://docs.microsoft.com/en-us/azure/devops/artifacts/concepts/feeds?view=azure-devops">Learn more about Artifacts Feed</a></p>
<h2 id="create-artifact-feed-in-azure-devops">Create Artifact Feed in Azure devops</h2>
<blockquote>
<p>Artifacts Feeds are organizational constructs that allow you to store, manage, and group your packages and control who to share it with. Feeds are not package-type dependent. You can store all the following package types in a single feed: npm, NuGet, Maven, Python, and Universal packages.</p>
</blockquote>
<ol>
<li>Go to Azure Artifacts.
<img src="https://i.imgur.com/DaCSK2x.png" alt="" class="full" /></li>
<li>Select Create Feed.
<img src="https://i.imgur.com/jST8Fj8.png" alt="" class="full" /></li>
<li>Give your feed a Name and choose its visibility, upstream sources and scope.
<img src="https://i.imgur.com/063qIt2.png" alt="" class="full" />
<img src="https://i.imgur.com/r4XYaCa.png" alt="" class="full" /></li>
</ol>
<h2 id="creating-azure-ci-pipeline">Creating Azure CI Pipeline</h2>
<p>Follow steps to create build definition</p>
<h3 id="step-1-create-template-yaml-file">Step 1: Create Template yaml File</h3>
<p>Then go to project and create install-node-module.yml file:
<img src="https://i.imgur.com/fLed9Hi.png" alt="" class="full" /></p>
<ol>
<li>We will install node.js</li>
<li>We will write the task to cache the node_modules folder into the azure feed.</li>
<li>Then we will install node packages if cache does not exist.</li>
</ol>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">task</span><span class="pi">:</span> <span class="s">NodeTool@0</span>
<span class="na">inputs</span><span class="pi">:</span>
<span class="na">versionSpec</span><span class="pi">:</span> <span class="s1">'</span><span class="s">14.x'</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s1">'</span><span class="s">Install</span><span class="nv"> </span><span class="s">Node.js'</span>
<span class="pi">-</span> <span class="na">task</span><span class="pi">:</span> <span class="s">1ESLighthouseEng.PipelineArtifactCaching.RestoreAndSaveCacheV1.RestoreAndSaveCache@1</span>
<span class="na">inputs</span><span class="pi">:</span>
<span class="na">keyfile</span><span class="pi">:</span> <span class="s1">'</span><span class="s">**/package-lock.json,</span><span class="nv"> </span><span class="s">!**/node_modules/**/package-lock.json,</span><span class="nv"> </span><span class="s">!**/.*/**/package-lock.json'</span>
<span class="na">targetfolder</span><span class="pi">:</span> <span class="s1">'</span><span class="s">$(Build.SourcesDirectory)/node_modules'</span>
<span class="na">vstsFeed</span><span class="pi">:</span> <span class="s1">'</span><span class="s">nx_azure_node_modules_cache_feed'</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">npm ci --ignore-engines</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">Install dependencies only if there is no cache available</span>
<span class="na">condition</span><span class="pi">:</span> <span class="s">ne(variables['CacheRestored'], 'true')</span>
</code></pre></div></div>
<h3 id="step-2-generate-the-ci-script-dynamically">Step 2: Generate the ci script dynamically</h3>
<p><img src="https://i.imgur.com/yRtHDzM.png" alt="" class="full" /></p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/**
* # Generate Ci command
* 1. It will identify the affected projects
* 2. Group the projects in 3 groups for each command.
* ## Example:
* If 3 projects are affected it will create below group for lint, test and build command
* lint1:[admin], lint2:[logger], lint3:[client]
* build1:[admin], build2:[logger], build3:[client]( if you made buildable for each projects)
* test1:[admin], test2:[logger], test3:[client]( if you have tests for each projects)
* If 4 projects are affected it will create below group for lint, test and build command
* lint1:[admin,ngmaterial], lint2:[logger], lint3:[client]
* build1:[admin,ngmaterial], build2:[logger], build3:[client]( if you made buildable for each projects)
* test1:[admin,ngmaterial], test2:[logger], test3:[client] ( if you have tests for each projects)
*/</span>
<span class="kd">const</span> <span class="nx">execSync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">child_process</span><span class="dl">'</span><span class="p">).</span><span class="nx">execSync</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">isMaster</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">===</span> <span class="dl">'</span><span class="s1">False</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">baseSha</span> <span class="o">=</span> <span class="nx">isMaster</span> <span class="p">?</span> <span class="dl">'</span><span class="s1">origin/main~1</span><span class="dl">'</span> <span class="p">:</span> <span class="dl">'</span><span class="s1">origin/main</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span>
<span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">lint</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">test</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">build</span><span class="dl">'</span><span class="p">),</span>
<span class="p">})</span>
<span class="p">);</span>
<span class="kd">function</span> <span class="nx">commands</span><span class="p">(</span><span class="nx">target</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">array</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span>
<span class="nx">execSync</span><span class="p">(</span><span class="s2">`npx nx print-affected --base=</span><span class="p">${</span><span class="nx">baseSha</span><span class="p">}</span><span class="s2"> --target=</span><span class="p">${</span><span class="nx">target</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">toString</span><span class="p">()</span>
<span class="p">.</span><span class="nx">trim</span><span class="p">()</span>
<span class="p">).</span><span class="nx">tasks</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">t</span><span class="p">)</span> <span class="o">=></span> <span class="nx">t</span><span class="p">.</span><span class="nx">target</span><span class="p">.</span><span class="nx">project</span><span class="p">);</span>
<span class="nx">array</span><span class="p">.</span><span class="nx">sort</span><span class="p">(()</span> <span class="o">=></span> <span class="mf">0.5</span> <span class="o">-</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">random</span><span class="p">());</span>
<span class="kd">const</span> <span class="nx">third</span> <span class="o">=</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">floor</span><span class="p">(</span><span class="nx">array</span><span class="p">.</span><span class="nx">length</span> <span class="o">/</span> <span class="mi">3</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">a1</span> <span class="o">=</span> <span class="nx">array</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">third</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">a2</span> <span class="o">=</span> <span class="nx">array</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="nx">third</span><span class="p">,</span> <span class="nx">third</span> <span class="o">*</span> <span class="mi">2</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">a3</span> <span class="o">=</span> <span class="nx">array</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="nx">third</span> <span class="o">*</span> <span class="mi">2</span><span class="p">);</span>
<span class="k">return</span> <span class="p">{</span>
<span class="p">[</span><span class="nx">target</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">1</span><span class="dl">'</span><span class="p">]:</span> <span class="nx">a1</span><span class="p">,</span>
<span class="p">[</span><span class="nx">target</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">2</span><span class="dl">'</span><span class="p">]:</span> <span class="nx">a2</span><span class="p">,</span>
<span class="p">[</span><span class="nx">target</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">3</span><span class="dl">'</span><span class="p">]:</span> <span class="nx">a3</span><span class="p">,</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="step--create-azure-pipelinesyml-file">Step : Create azure-pipelines.yml file</h3>
<p><img src="https://i.imgur.com/2oCIB4J.png" alt="" class="full" /></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">trigger</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">main</span> <span class="c1"># Trigger CI automatically whenever main branch is changed</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">initial_setup</span> <span class="c1"># Install Node.js & Node Packages & Generate Commands for Affected Projects.</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">IS_PR</span><span class="pi">:</span> <span class="s">$[ eq(variables['Build.Reason'], 'PullRequest') ]</span> <span class="c1"># Findout Is this pull request?</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span> <span class="c1"># Base Template</span>
<span class="pi">-</span> <span class="na">powershell</span><span class="pi">:</span> <span class="s">echo "##vso[task.setvariable variable=COMMANDS;isOutput=true]$(node ./tools/scripts/generate-ci-commands.js $(IS_PR))"</span> <span class="c1"># Create COMMANDS build time variable and assign ci commands.</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">setCommands</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">echo $(setCommands.COMMANDS)</span> <span class="c1"># Echo the command for log purpose.</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">echoCommands</span>
</code></pre></div></div>
<h3 id="step--lets-run-the-build">Step : Lets Run the Build</h3>
<p><img src="https://i.imgur.com/GOUP7Se.png" alt="" class="full" /></p>
<p>Select run
<img src="https://i.imgur.com/sO8rVJ0.png" alt="" class="full" /></p>
<p>This time we have not changed the code therefore no build command created.
<img src="https://i.imgur.com/JxNUOsa.png" alt="" class="full" /></p>
<h3 id="step--create-run-many-javascript">Step : Create Run Many JavaScript</h3>
<p><img src="https://i.imgur.com/bEu4amr.png" alt="" class="full" /></p>
<p>In order to run the lint, test and build we have to create run-many.js file where we will put our script.</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/**
* # Run Many
* It will run the script using nx command line to run them in parellel.
*/</span>
<span class="kd">const</span> <span class="nx">execSync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">child_process</span><span class="dl">'</span><span class="p">).</span><span class="nx">execSync</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">commands</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]);</span>
<span class="kd">const</span> <span class="nx">projects</span> <span class="o">=</span> <span class="nx">commands</span><span class="p">[</span><span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">3</span><span class="p">]];</span>
<span class="kd">const</span> <span class="nx">target</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">4</span><span class="p">];</span>
<span class="nx">execSync</span><span class="p">(</span>
<span class="s2">`npx nx run-many --target=</span><span class="p">${</span><span class="nx">target</span><span class="p">}</span><span class="s2"> --projects=</span><span class="p">${</span><span class="nx">projects</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span>
<span class="dl">'</span><span class="s1">,</span><span class="dl">'</span>
<span class="p">)}</span><span class="s2"> --parallel`</span><span class="p">,</span>
<span class="p">{</span>
<span class="na">stdio</span><span class="p">:</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
<span class="p">}</span>
<span class="p">);</span>
</code></pre></div></div>
<h3 id="step--add-lint-test-and-build-scripts">Step : Add lint, test and build scripts</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">trigger</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">main</span> <span class="c1"># Trigger CI automatically whenever main branch is changed</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">initial_setup</span> <span class="c1"># Install Node.js & Node Packages & Generate Commands for Affected Projects.</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">IS_PR</span><span class="pi">:</span> <span class="s">$[ eq(variables['Build.Reason'], 'PullRequest') ]</span> <span class="c1"># Findout Is this pull request?</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="s">CAMTASIA_19_MEDIA_FORMAT*C:\Users\Rupesh\Documents\Camtasia\Rec 01-21-21_003.trec|Product - template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span> <span class="c1"># Base Template</span>
<span class="pi">-</span> <span class="na">powershell</span><span class="pi">:</span> <span class="s">echo "##vso[task.setvariable variable=COMMANDS;isOutput=true]$(node ./tools/scripts/generate-ci-commands.js $(IS_PR))"</span> <span class="c1"># Create COMMANDS build time variable and assign ci commands.</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">setCommands</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">echo $(setCommands.COMMANDS)</span> <span class="c1"># Echo the command for log purpose.</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">echoCommands</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">lint1</span> <span class="c1"># Lets Run the Linting command for lint1 array.</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span>
<span class="pi">|</span> <span class="c1"># Make sure this is passing and Don't Run this if we have empty array in nx commands from intial_setup step.</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"lint1":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' lint1 lint</span> <span class="c1"># Run the lint command in parallel</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">lint2</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"lint2":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' lint2 lint</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">lint3</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"lint2":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' lint3 lint</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">test1</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"test1":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' test1 test</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">test2</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"test2":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' test2 test</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">test3</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"test3":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' test3 test</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">build1</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"build1":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' build1 build`</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">build2</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"build2":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' build2 build</span>
<span class="pi">-</span> <span class="na">job</span><span class="pi">:</span> <span class="s">build3</span>
<span class="na">dependsOn</span><span class="pi">:</span> <span class="s">initial_setup</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">dependencies.initial_setup.outputs['setCommands.COMMANDS'],</span>
<span class="s">'"build3":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="na">pool</span><span class="pi">:</span>
<span class="na">vmImage</span><span class="pi">:</span> <span class="s1">'</span><span class="s">ubuntu-latest'</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">COMMANDS</span><span class="pi">:</span> <span class="s">$[ dependencies.initial_setup.outputs['setCommands.COMMANDS'] ]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">template</span><span class="pi">:</span> <span class="s">.azure-pipelines/steps/install-node-modules.yml</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' build3 build</span>
</code></pre></div></div>
<h3 id="step--lets-run-the-build-again">Step : Lets Run the build Again</h3>
<p>Notice since we have not changed any code we will notice only initial_setup and rest all jobs will be skipped.</p>
<p><img src="https://i.imgur.com/QiHKC9V.png" alt="" class="full" /></p>
<h3 id="step--lets-change-the-logger-and-find-out-affected-dependency-graph">Step : Lets Change the Logger and find out affected dependency graph</h3>
<p>Go libs\branding\logger\src\lib\logging.service.ts
<img src="https://i.imgur.com/aeU11m5.png" alt="" class="full" /></p>
<p>Run <code class="language-plaintext highlighter-rouge">npm run affected:dep-graph</code></p>
<p><img src="https://i.imgur.com/Tu2E3ML.png" alt="" class="full" /></p>
<p>confirm changes are working Run <code class="language-plaintext highlighter-rouge">npm run start-admin</code></p>
<p><img src="https://i.imgur.com/iVTsegC.png" alt="" class="full" /></p>
<h3 id="step-fix-admin-specs">Step: Fix admin specs</h3>
<p>Lets create a script to test and watch
<img src="https://i.imgur.com/FTvQoP8.png" alt="" class="full" /></p>
<p>Now lets fix the test for admin project correct the title name
Now run <code class="language-plaintext highlighter-rouge">npm run affected:test</code></p>
<p>It will automatically run the admin test.</p>
<p><img src="https://i.imgur.com/zxF2qE5.png" alt="" class="full" /></p>
<h3 id="step-fix-client-specs">Step: Fix client specs</h3>
<p>Now lest fix the test for client project correct the title name
Now run <code class="language-plaintext highlighter-rouge">npm run affected:test</code>
It will automatically run the both admin & client test.
<img src="https://i.imgur.com/WAnxb3L.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/8tn43Vi.png" alt="" class="full" /></p>
<h3 id="step--lets-push-the-code-and-notice-our-builds">Step : Lets Push the code and notice our Builds</h3>
<p>See this time in the build machine as well it identified</p>
<ol>
<li>Only 2 projects to lint and test and they are
<ol>
<li>“branding-logger”,”cutepuppies-admin”</li>
</ol>
</li>
<li>Only 1 Project to build and that is:”cutepuppies-admin”</li>
</ol>
<p>Notice test and build are passing</p>
<p><img src="https://i.imgur.com/Yj8uQdv.png" alt="" class="full" /></p>
<h3 id="step--running-all-lint-build-and-test-as-task">Step : Running all lint build and test as TASK</h3>
<p>We dont want to trigger these many jobs since our monorepo is not that large. Therefore, I am deciding to create just one job so lets follow below steps.</p>
<h3 id="step-update-the-command-generation-script">Step: Update the command generation script</h3>
<p>Lets update the build definition to run them as task within single job.</p>
<p>Rename the current file to generate-ci-batch-commands.js
And create new file generate-ci-commands.js
<img src="https://i.imgur.com/RK1V28L.png" alt="" class="full" /></p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/**
* # Generate Ci command
* 1. It will identify the affected projects
* 2. Group the projects in 3 groups for each command.
* ## Example:
* If 3 projects are affected it will create below group for lint, test and build command
* lint1:[admin], lint2:[logger], lint3:[client]
* build1:[admin], build2:[logger], build3:[client]( if you made buildable for each projects)
* test1:[admin], test2:[logger], test3:[client]( if you have tests for each projects)
* If 4 projects are affected it will create below group for lint, test and build command
* lint1:[admin,ngmaterial], lint2:[logger], lint3:[client]
* build1:[admin,ngmaterial], build2:[logger], build3:[client]( if you made buildable for each projects)
* test1:[admin,ngmaterial], test2:[logger], test3:[client] ( if you have tests for each projects)
*/</span>
<span class="k">debugger</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">execSync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">child_process</span><span class="dl">'</span><span class="p">).</span><span class="nx">execSync</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">isMaster</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">===</span> <span class="dl">'</span><span class="s1">False</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">baseSha</span> <span class="o">=</span> <span class="nx">isMaster</span> <span class="p">?</span> <span class="dl">'</span><span class="s1">origin/main~1</span><span class="dl">'</span> <span class="p">:</span> <span class="dl">'</span><span class="s1">origin/main</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span>
<span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">lint</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">test</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">build</span><span class="dl">'</span><span class="p">),</span>
<span class="p">})</span>
<span class="p">);</span>
<span class="kd">function</span> <span class="nx">commands</span><span class="p">(</span><span class="nx">target</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">array</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span>
<span class="nx">execSync</span><span class="p">(</span><span class="s2">`npx nx print-affected --base=</span><span class="p">${</span><span class="nx">baseSha</span><span class="p">}</span><span class="s2"> --target=</span><span class="p">${</span><span class="nx">target</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">toString</span><span class="p">()</span>
<span class="p">.</span><span class="nx">trim</span><span class="p">()</span>
<span class="p">).</span><span class="nx">tasks</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">t</span><span class="p">)</span> <span class="o">=></span> <span class="nx">t</span><span class="p">.</span><span class="nx">target</span><span class="p">.</span><span class="nx">project</span><span class="p">);</span>
<span class="k">return</span> <span class="p">{</span> <span class="p">[</span><span class="nx">target</span><span class="p">]:</span> <span class="nx">array</span> <span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If you run this script, after changing logger project. It will generate the commands for the affected projects. Like below</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"lint"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-admin"</span><span class="p">,</span><span class="w"> </span><span class="s2">"branding-logger"</span><span class="p">],</span><span class="w">
</span><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-admin"</span><span class="p">,</span><span class="w"> </span><span class="s2">"branding-logger"</span><span class="p">],</span><span class="w">
</span><span class="nl">"build"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-admin"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p><img src="https://i.imgur.com/IiVl5jO.png" alt="" class="full" /></p>
<h3 id="step--update-the-azure-pipeline-yml-to-have-steps-not-jobs">Step : Update the azure-pipeline yml to have Steps not jobs</h3>
<p>Rename the current azure-pipeline yml
<img src="https://i.imgur.com/tNAGfrK.png" alt="" class="full" /></p>
<p>and move to .azure-pipeline/steps folder for future use or backup.</p>
<p><img src="https://i.imgur.com/67QekSa.png" alt="" class="full" /></p>
<p>azure-pipelines.yml</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">trigger</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">main</span> <span class="c1"># Trigger CI automatically whenever main branch is changed</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">IS_PULLREQUEST</span><span class="pi">:</span> <span class="s">$[eq(variables['Build.Reason'], 'PullRequest')]</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">task</span><span class="pi">:</span> <span class="s">Cache@2</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s1">'</span><span class="s">Cache</span><span class="nv"> </span><span class="s">Npm</span><span class="nv"> </span><span class="s">Dependencies'</span>
<span class="na">inputs</span><span class="pi">:</span>
<span class="na">key</span><span class="pi">:</span> <span class="s1">'</span><span class="s">**/package-lock.json,</span><span class="nv"> </span><span class="s">!**/node_modules/**/package-lock.json,</span><span class="nv"> </span><span class="s">!**/.*/**/package-lock.json'</span>
<span class="na">path</span><span class="pi">:</span> <span class="s1">'</span><span class="s">$(Build.SourcesDirectory)/node_modules'</span>
<span class="na">cacheHitVar</span><span class="pi">:</span> <span class="s1">'</span><span class="s">CacheRestored'</span>
<span class="pi">-</span> <span class="na">task</span><span class="pi">:</span> <span class="s">Npm@1</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s1">'</span><span class="s">npm</span><span class="nv"> </span><span class="s">ci'</span>
<span class="na">inputs</span><span class="pi">:</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">ci</span>
<span class="na">verbose</span><span class="pi">:</span> <span class="no">false</span>
<span class="na">condition</span><span class="pi">:</span> <span class="s">ne(variables['CacheRestored'], 'true')</span>
<span class="pi">-</span> <span class="na">powershell</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">echo "##vso[task.setvariable variable=COMMANDS]$(node ./tools/scripts/generate-ci-commands.js $(IS_PULLREQUEST))"</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">setCommands</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">setting commands</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">echo $(COMMANDS)</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">echoCommands</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">commands to run</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' lint</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">linting</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">variables['COMMANDS'],</span>
<span class="s">'"lint":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' test</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">testing</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">variables['COMMANDS'],</span>
<span class="s">'"test":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
<span class="pi">-</span> <span class="na">script</span><span class="pi">:</span> <span class="s">node ./tools/scripts/run-many.js '$(COMMANDS)' build</span>
<span class="na">displayName</span><span class="pi">:</span> <span class="s">building</span>
<span class="na">condition</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">and(</span>
<span class="s">succeeded(),</span>
<span class="s">not(contains(</span>
<span class="s">variables['COMMANDS'],</span>
<span class="s">'"build":[]'</span>
<span class="s">))</span>
<span class="s">)</span>
</code></pre></div></div>
<p>Run build</p>
<p><img src="https://i.imgur.com/Pjkjjwu.png" alt="" class="full" /></p>
<h3 id="step--change-logging-service-only-and-run-build">Step : Change logging service only and run build</h3>
<p><img src="https://i.imgur.com/tWw04V4.png" alt="" class="full" /></p>
<p>Run build</p>
<p><img src="https://i.imgur.com/IGmdmo1.png" alt="" class="full" /></p>
<p>Build Success
<img src="https://i.imgur.com/8F5gXDb.png" alt="" class="full" /></p>
<h3 id="step--finding-affected-apps">Step : Finding affected apps</h3>
<p>Lets update the commands to also give a list of apps to publish in npm repository.</p>
<p>When We change logger we want to also see publish array with affected app.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"lint"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"branding-logger"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"branding-logger"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"build"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"publish"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-admin"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>If I change the client app as well then I should see both apps in publish array.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"lint"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-client"</span><span class="p">,</span><span class="w"> </span><span class="s2">"branding-logger"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-client"</span><span class="p">,</span><span class="w"> </span><span class="s2">"branding-logger"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"build"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-client"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">],</span><span class="w">
</span><span class="nl">"publish"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"cutepuppies-client"</span><span class="p">,</span><span class="w"> </span><span class="s2">"cutepuppies-admin"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>I will add below method for getting publish apps</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">affectedApps</span><span class="p">(</span><span class="nx">command</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">x</span> <span class="o">=</span> <span class="nx">execSync</span><span class="p">(</span><span class="s2">`npx nx affected:apps --base=</span><span class="p">${</span><span class="nx">baseSha</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">toString</span><span class="p">()</span>
<span class="p">.</span><span class="nx">trim</span><span class="p">();</span>
<span class="kd">let</span> <span class="nx">apps</span> <span class="o">=</span> <span class="nx">x</span> <span class="p">?</span> <span class="nx">x</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="se">\n\n</span><span class="dl">'</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="s1"> - </span><span class="dl">'</span><span class="p">).</span><span class="nx">slice</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">:</span> <span class="p">[];</span>
<span class="nx">apps</span> <span class="o">=</span> <span class="nx">apps</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">t</span><span class="p">)</span> <span class="o">=></span> <span class="nx">t</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">,</span> <span class="dl">''</span><span class="p">).</span><span class="nx">trim</span><span class="p">());</span>
<span class="k">return</span> <span class="p">{</span> <span class="p">[</span><span class="nx">command</span><span class="p">]:</span> <span class="nx">apps</span> <span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The new script for <code class="language-plaintext highlighter-rouge">generate-ci-command.js</code></p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">execSync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">child_process</span><span class="dl">'</span><span class="p">).</span><span class="nx">execSync</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">isMaster</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">===</span> <span class="dl">'</span><span class="s1">False</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">baseSha</span> <span class="o">=</span> <span class="nx">isMaster</span> <span class="p">?</span> <span class="dl">'</span><span class="s1">origin/main~1</span><span class="dl">'</span> <span class="p">:</span> <span class="dl">'</span><span class="s1">origin/main</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span>
<span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">lint</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">test</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">commands</span><span class="p">(</span><span class="dl">'</span><span class="s1">build</span><span class="dl">'</span><span class="p">),</span>
<span class="p">...</span><span class="nx">affectedApps</span><span class="p">(</span><span class="dl">'</span><span class="s1">publish</span><span class="dl">'</span><span class="p">),</span> <span class="c1">// publish command for the apps</span>
<span class="p">})</span>
<span class="p">);</span>
<span class="kd">function</span> <span class="nx">commands</span><span class="p">(</span><span class="nx">target</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">array</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span>
<span class="nx">execSync</span><span class="p">(</span><span class="s2">`npx nx print-affected --base=</span><span class="p">${</span><span class="nx">baseSha</span><span class="p">}</span><span class="s2"> --target=</span><span class="p">${</span><span class="nx">target</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">toString</span><span class="p">()</span>
<span class="p">.</span><span class="nx">trim</span><span class="p">()</span>
<span class="p">).</span><span class="nx">tasks</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">t</span><span class="p">)</span> <span class="o">=></span> <span class="nx">t</span><span class="p">.</span><span class="nx">target</span><span class="p">.</span><span class="nx">project</span><span class="p">);</span>
<span class="k">return</span> <span class="p">{</span> <span class="p">[</span><span class="nx">target</span><span class="p">]:</span> <span class="nx">array</span> <span class="p">};</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">affectedApps</span><span class="p">(</span><span class="nx">command</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">x</span> <span class="o">=</span> <span class="nx">execSync</span><span class="p">(</span><span class="s2">`npx nx affected:apps --base=</span><span class="p">${</span><span class="nx">baseSha</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="p">.</span><span class="nx">toString</span><span class="p">()</span>
<span class="p">.</span><span class="nx">trim</span><span class="p">();</span>
<span class="kd">let</span> <span class="nx">apps</span> <span class="o">=</span> <span class="nx">x</span> <span class="p">?</span> <span class="nx">x</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="se">\n\n</span><span class="dl">'</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="s1"> - </span><span class="dl">'</span><span class="p">).</span><span class="nx">slice</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">:</span> <span class="p">[];</span>
<span class="nx">apps</span> <span class="o">=</span> <span class="nx">apps</span><span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">t</span><span class="p">)</span> <span class="o">=></span> <span class="nx">t</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">,</span> <span class="dl">''</span><span class="p">).</span><span class="nx">trim</span><span class="p">());</span>
<span class="k">return</span> <span class="p">{</span> <span class="p">[</span><span class="nx">command</span><span class="p">]:</span> <span class="nx">apps</span> <span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="step--lets-add-packagejson-in-applications">Step : Lets Add Package.json in applications</h3>
<p>Admin app package.json
<img src="https://i.imgur.com/FBImW9k.png" alt="" class="full" /></p>
<p>Client app package.json</p>
<p><img src="https://i.imgur.com/dsaUa6N.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/FtHLQEb.png" alt="" class="full" /></p>
<h3 id="step--update-workspacejson-for-app">Step : Update workspace.json for app</h3>
<p>In order to copy package.json file after build in the dist folder update the workspace file.</p>
<p>admin project
<img src="https://i.imgur.com/Wksie6X.png" alt="" class="full" /></p>
<p>client project</p>
<p><img src="https://i.imgur.com/FiXMBLL.png" alt="" class="full" /></p>
<h3 id="step--build-in-production-mode">Step : Build in Production Mode</h3>
<p>I want to build my angular app with below</p>
<ol>
<li>production mode</li>
<li>no output file hashing</li>
<li>I want source map file so that I can debug my source file.</li>
</ol>
<p>Below is the nx build command to fulfill above requirements</p>
<pre><code class="language-shell=">nx run-many --target=build --projects=cutepuppies-client,cutepuppies-admin --parallel --configuration=production --outputHashing=none --sourceMap=true
</code></pre>
<p>Run above nx build command</p>
<p><img src="https://i.imgur.com/qFdZhxH.png" alt="" class="full" /></p>
<p>Notice in the dist folder in admin folder we got map file and package.json
<img src="https://i.imgur.com/xZq2B3J.png" alt="" class="full" />
client project
<img src="https://i.imgur.com/0v10ELf.png" alt="" class="full" /></p>
<h3 id="step-publishing-test-results-in-azure-pipeline-build">Step: Publishing Test Results in Azure Pipeline Build</h3>
<p>https://hackmd.io/ChTI9iFXQXGdlQAWrcqrKA</p>
<h3 id="step-publishing-code-coverage-in-azure-pipeline-build">Step: Publishing Code Coverage in Azure Pipeline Build</h3>
<p>https://hackmd.io/Aiksk5NqQouDgKcZLjnHfQ</p>
<h3 id="step-add-npmrc-file-to-app-that-we-want-to-publish-npm">Step: Add .npmrc file to app that we want to publish npm</h3>
<p>Get access token from (npm.js)[https://www.npmjs.com/]</p>
<p><img src="https://i.imgur.com/XIRXjjC.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/ZXEZZYf.png" alt="" class="full" /></p>
<p>Generating new access token in npmjs</p>
<p><img src="https://i.imgur.com/VlwX0n3.png" alt="" class="full" /></p>
<p>Adding npmrc file with authtoken
<img src="https://i.imgur.com/cHRfq8B.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/aKO5rIy.png" alt="" class="full" /></p>
<h3 id="step-updating-packagejson-version--publish-npm-package">Step: Updating Package.json Version & Publish Npm package</h3>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/**
* . bump the version
* . add npmrc file (if required)
* . publish to npm package repository
*
* Example: node ./publish-npm.js $(COMMANDS) $(Build.BuildNumber)
*
*/</span>
<span class="kd">const</span> <span class="nx">execSync</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">child_process</span><span class="dl">'</span><span class="p">).</span><span class="nx">execSync</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">fs</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">path</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">path</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">basePackageJson</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">../../package.json</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">args</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">commands</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">],</span>
<span class="na">buildNumber</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">3</span><span class="p">],</span>
<span class="p">};</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Received Args: </span><span class="dl">'</span><span class="p">,</span> <span class="nx">args</span><span class="p">,</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">args</span><span class="p">.</span><span class="nx">commands</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Commands are required</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">commands</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">args</span><span class="p">.</span><span class="nx">commands</span><span class="p">);</span>
<span class="cm">/*
let commands = {
publish: ['cutepuppies-admin', 'cutepuppies-client'],
};
*/</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">args</span><span class="p">.</span><span class="nx">buildNumber</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Build Number is required</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">buildNumber</span> <span class="o">=</span> <span class="nx">args</span><span class="p">.</span><span class="nx">buildNumber</span><span class="p">.</span><span class="nx">toString</span><span class="p">();</span>
<span class="c1">// let buildNumber = '3243';</span>
<span class="kd">const</span> <span class="nx">projects</span> <span class="o">=</span> <span class="nx">commands</span><span class="p">[</span><span class="dl">'</span><span class="s1">publish</span><span class="dl">'</span><span class="p">];</span>
<span class="kd">const</span> <span class="nx">newVersion</span> <span class="o">=</span> <span class="nx">getNewVesrion</span><span class="p">();</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`new npm version would be: </span><span class="p">${</span><span class="nx">newVersion</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">updatePublishingPackageJsonVersion</span><span class="p">();</span>
<span class="nx">publishNpmPackage</span><span class="p">();</span>
<span class="k">return</span> <span class="nx">newVersion</span><span class="p">;</span>
<span class="kd">function</span> <span class="nx">getNewVesrion</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">let</span> <span class="nx">currentVersion</span> <span class="o">=</span> <span class="nx">basePackageJson</span><span class="p">.</span><span class="nx">version</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">currentVersion</span>
<span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="s1">.</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">map</span><span class="p">((</span><span class="nx">x</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=></span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">2</span> <span class="p">?</span> <span class="nx">buildNumber</span> <span class="p">:</span> <span class="nx">x</span><span class="p">))</span>
<span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="dl">'</span><span class="s1">.</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">updatePublishingPackageJsonVersion</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">projects</span><span class="p">.</span><span class="nx">forEach</span><span class="p">((</span><span class="nx">project</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">updateVersion</span><span class="p">(</span>
<span class="nx">path</span><span class="p">.</span><span class="nx">resolve</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="dl">'</span><span class="s1">../../</span><span class="dl">'</span><span class="p">,</span> <span class="s2">`dist/apps/</span><span class="p">${</span><span class="nx">project</span><span class="p">}</span><span class="s2">/package.json`</span><span class="p">)</span>
<span class="p">);</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">updateVersion</span><span class="p">(</span><span class="nx">packageJsonFilePath</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="kr">package</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="nx">packageJsonFilePath</span><span class="p">);</span>
<span class="kr">package</span><span class="p">.</span><span class="nx">version</span> <span class="o">=</span> <span class="nx">newVersion</span><span class="p">;</span>
<span class="nx">fs</span><span class="p">.</span><span class="nx">writeFileSync</span><span class="p">(</span><span class="nx">packageJsonFilePath</span><span class="p">,</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="kr">package</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">));</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span>
<span class="s2">`Version updated for the app </span><span class="p">${</span><span class="kr">package</span><span class="p">.</span><span class="nx">name</span><span class="p">}</span><span class="s2">: </span><span class="p">${</span><span class="nx">newVersion</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
<span class="dl">'</span><span class="se">\n</span><span class="dl">'</span>
<span class="p">);</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">publishNpmPackage</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">projects</span><span class="p">.</span><span class="nx">forEach</span><span class="p">((</span><span class="nx">app</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">cwd</span> <span class="o">=</span> <span class="nx">path</span><span class="p">.</span><span class="nx">resolve</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="dl">'</span><span class="s1">../../</span><span class="dl">'</span><span class="p">,</span> <span class="s2">`dist/apps/</span><span class="p">${</span><span class="nx">app</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`publishing to npm from: `</span><span class="p">,</span> <span class="nx">cwd</span><span class="p">,</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">execSync</span><span class="p">(</span>
<span class="s2">`npm publish --access public`</span><span class="p">,</span>
<span class="p">{</span> <span class="nx">cwd</span><span class="p">,</span> <span class="na">stdio</span><span class="p">:</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span> <span class="p">},</span>
<span class="kd">function</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">);</span>
<span class="k">throw</span> <span class="nx">error</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`</span><span class="p">${</span><span class="nx">app</span><span class="p">}</span><span class="s2"> is published`</span><span class="p">,</span> <span class="nx">newVersion</span><span class="p">,</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="step--push-you-changes-and-trigger-build">Step : Push you changes and Trigger Build</h3>
<p>See both admin and client is published.
<img src="https://i.imgur.com/yd4fgCr.png" alt="" class="full" /></p>
<h3 id="step-only-change-admin-and-notice-only-admin-is-published-in-npm">Step: Only change Admin and Notice only Admin is published in NPM</h3>
<p><img src="https://i.imgur.com/vxX7r3e.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/WKNOavp.png" alt="" class="full" /></p>
<h2 id="karma-failing-when-no-test-present">Karma failing when no test present</h2>
<p>Karma test will fail if u have not tests.
Therefore, if you want to stop this behavior then you need to go to <code class="language-plaintext highlighter-rouge">karma.conf.js</code> at root location where u see <code class="language-plaintext highlighter-rouge">package.json</code> and add below flag as false.</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w"> </span><span class="err">failOnEmptyTestSuite:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span></code></pre></div></div>
<h2 id="monorepo-architecture-guideline-for-developers">Monorepo Architecture Guideline for developers</h2>
<h3 id="what-do-smart-components-do">What do smart components do? </h3>
<ul>
<li>It can do dependency injection and have service stores injected.</li>
<li>It can only send command</li>
<li>It can only call query methods to store to get some result/data</li>
<li>It can not write any filter,sort kind of logic move them in store</li>
<li>It can not publish event</li>
<li>It can not have business query logic or model update logic</li>
<li>It can pass some data structure to nested dumb/presentation components</li>
</ul>
<h3 id="what-is-dumb--presentation-componetn-can-do">What is dumb / presentation componetn can do?</h3>
<ul>
<li>No dependency injection for service and store</li>
<li>Only input and output</li>
<li>Responsible for rendering data</li>
<li>No message publishing/sending</li>
<li>No store query no model state mutation</li>
</ul>
<h3 id="what-is-a-service">What is a service?</h3>
<ol>
<li>It has business logic and</li>
<li>It can take advantage of API service.</li>
<li>Most of the services are message handlers.</li>
</ol>
<p>Quiz: Where should I write the code to create a mcq widget?</p>
<h3 id="what-is-a-model">What is a Model?</h3>
<ul>
<li>
<p>It is a data structure which has state.</p>
</li>
<li>It must not have business logic.</li>
<li>It can have getters and setters.</li>
</ul>
<h3 id="what-is-a-store">What is a Store? </h3>
<p>Is a collection of models. If you have multiple models of the same type, create store to retrieve them and put them in the collection.</p>
<ol>
<li>Don’t write business logic to mutate the model state in store</li>
<li>Business logic to mutate model state goes in SERVICES only.</li>
<li>All Model related queries (filter, sort etc ) must be written in the Store that component can call upon.</li>
<li>
<p>Store we can write code that does not go in one model and you want span over the multiple instances of the model.</p>
</li>
<li>Example: Sorting models inside store, Asserting before inserting model in collection.</li>
</ol>
<h3 id="what-is-an-api-service">What is an API service?</h3>
<ol>
<li>Makes http network calls and returns promises.</li>
<li>It has httpClient dependency and extends Base API Service.</li>
<li>API service mostly returns the client side Model object and Server is supposed to return the same Model Data Structure to the client. However, in case there is a mismatch between the result comes from Server vs client. Then Use DTO for server side data structure. DTO should remain under the DATA folder of the project.</li>
</ol>
<h3 id="who-depends-on-whom-within-a-library-">Who depends on whom within a library ? </h3>
<p>We have component layer, message handlers and api service layers.</p>
<ul>
<li>Component can depend on Model and Message service</li>
<li>Message Handler can depend on model and api</li>
<li>API service can depend on httpclient and dto</li>
</ul>
<h3 id="before-push-to-git-guidelines">Before Push to Git Guidelines</h3>
<p>✅ RUN TEST: npm run affected:test
✅ RUN LINT: npm run affected:lint</p>
<p>You can always run affected projects on your feature branch by giving its name.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm run affected:test --base=origin/features/PBI23/MessageHandlers
npm run affected:lint --base=origin/features/PBI23/MessageHandlers
</code></pre></div></div>
<h3 id="naming-conventions">Naming conventions</h3>
<p>Feel free to follow any naming conventions provided in your organization. Here is what I recommend:</p>
<p><img src="https://i.imgur.com/Z7x7ZUP.png" alt="" class="full" /></p>
<h3 id="nx-related-questions">Nx Related Questions</h3>
<ul>
<li>
<p><strong>Does the lib projects have npm versions?</strong>
<strong>Answer:</strong> No, because they are not buildable and publishable.</p>
</li>
<li>
<p><strong>How should we branch in monorepo ?</strong>
<strong>Answer:</strong> V1, V2, V3 are the branches for the monorepo.</p>
</li>
<li>
<p><strong>When do we create branches ?</strong>
<strong>Answer:</strong> As you move your code to a new environment it is recommended to create a separate branch in monorepo.</p>
</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><strong>Source Code</strong>: https://github.com/rupeshtiwari/coding-examples-angular-monorepo-with-nx</li>
<li><strong>Source Code</strong>: https://github.com/rupeshtiwari/coding-examples-angular-monorepo-nx-azure-ci-cd</li>
<li><a href="https://ss64.com/nt/">An A-Z Index of the Windows CMD command line</a></li>
<li>https://nx.dev/latest/react/getting-started/getting-started</li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" class="full" /></p>Rupesh TiwariDo you have a lot of JavaScript projects using GitHub repositories? How are you managing dependencies among those projects? Developers might be struggling to manage a large set of GitHub repositories and Business might be spending a lot of money to manage them. The potential solution to this problem is Monorepo. You can put all of your JavaScript projects regardless of framework (angular, vue.js, react.js) in a single GitHub repository called monorepo. This will solve the problem and you will not have many Git projects. However, nothing is free! So there are many challenges with monorepo that developers are facing. I will explain all of the challenges and show you how you can solve them. Also I will create monorepo from scratch in this article using angular projects as samples.Symmetric and Asymmetric Encryption Basics2021-11-19T00:00:00+00:002021-11-19T00:00:00+00:00https://www.rupeshtiwari.com/symmetric-and-asymmetric-encryption-basics<blockquote>
<p>Why do you care about encryption strategies? Well if you are planning to become system administrator or solution architect then you must know how you would secure your applications deployed on-premise or cloud. Also remember this question is asked in most of the solution architect interviews.</p>
</blockquote>
<h2 id="why-do-you-need-to-encrypt-user-data">Why do you need to encrypt user data?</h2>
<p>Suppose you have a banking website hosted on a server. A user logged in to your application and he is fetching his account details. If the server does not encrypt the data sent over the internet to the user machine then any attacker can sniff the traffic and steal the account information. Also possible some attacker can steal the cookie or session key and authenticate himself as you and start doing transactions on behalf of you called <a href="https://owasp.org/www-community/attacks/Session_fixation">session fixation</a>. There is also the possibility of cross site scripting (XSS) and cross site request forgery (XSRF) attacks which are in top <a href="https://owasp.org/www-project-top-ten/">10 vulnerability attacks identified by Open Web Application Security Project (OWASP</a>) if you do not secure your communication between server application and client.</p>
<p><img src="https://i.imgur.com/8jIjVJJ.png" alt="" class="full" /></p>
<p>I hope no you are convinced that you must learn encryption mechanisms in order to protect your server.</p>
<h2 id="what-is-cryptography">What is cryptography?</h2>
<p>Cryptography is the study of secure communications techniques that allow only the sender and intended recipient of a message to view its contents. The term is derived from the Greek word kryptos, which means hidden. - <a href="https://www.kaspersky.com/resource-center/definitions/what-is-cryptography">Cryptography Definitions</a></p>
<h2 id="what-is-symmetric-encryption">What is Symmetric Encryption?</h2>
<p>In the symmetric encryption you use a single key to encrypt and decrypt user data. Both server (sender) and client (receiver) must have the same key in order for sending and receiving data successfully.</p>
<p><img src="https://i.imgur.com/t2Fs7Xa.png" alt="" class="full" /></p>
<p><a href="https://en.wikipedia.org/wiki/Transport_Layer_Security">TLS/SSL</a> uses symmetric cryptography using the session key after the initial handshake is done. The most frequently and widely used symmetric encryption algorithms are AES-128, AES-192 and AES-256.</p>
<h2 id="what-is-asymmetric-encryption">What is Asymmetric encryption?</h2>
<p>In Asymmetric encryption the sender creates a pair of keys. One is called a public key, another one is a private key. The user data is encrypted by using a private key which can only be decrypted by using the corresponding public key.</p>
<p><img src="https://i.imgur.com/Pr8xBQv.png" alt="" class="full" /></p>
<p>TLS/SSL uses asymmetric cryptography to initiate the communication which is also known as TLS/SSL handshake. Most widely used asymmetric encryption algorithms are elgamal, RSA, Elliptic curve technique and PKCS.</p>Rupesh TiwariWhy do you care about encryption strategies? Well if you are planning to become system administrator or solution architect then you must know how you would secure your applications deployed on-premise or cloud. Also remember this question is asked in most of the solution architect interviews.Azure Service Endpoint Overview and Configuration2021-11-17T00:00:00+00:002021-11-17T00:00:00+00:00https://www.rupeshtiwari.com/azure-service-endpoint-and-private-endpoint-overview-and-configuration<blockquote>
<p>Did you know By default azure storage accounts are accessible by the public internet. How can u restrict all public traffic in to your azure storage account and only allow your VNet resources to connect it. Lets learn the same in this article.</p>
</blockquote>
<h2 id="how-to-access-blob-storage-from-vm-by-minimizing-internet-access">How to access blob storage from VM by minimizing internet access?</h2>
<p>Suppose we have one VM om Azure where we have deployed our web server that is hosting rupeshtiwari.com. Now I want to show some images which I want to bring from my azure Blob storage account.</p>
<p>By default all of my images in the azure storage account is accessible from internet. Because, Storage account has <code class="language-plaintext highlighter-rouge">allow access from all network</code> as default settings.</p>
<p><img src="https://i.imgur.com/W7wbOIK.png" alt="" class="full" /></p>
<p>So question is how can I restrict all internet access to my blob storage and only allow my webserver to communicate to blob storage?</p>
<p><img src="https://i.imgur.com/Ex7bJAW.png" alt="" class="full" /></p>
<p><strong>Step 1:</strong> <strong>Create Public IP Address (104.41.138.18) and allow only this IP to access blob storage</strong> and restrict internet access in blob storage. You can do this in the Azure portal by going <code class="language-plaintext highlighter-rouge">Firewall and Virtual Network</code> section of blob storage.</p>
<p><img src="https://i.imgur.com/PddFfJo.png" alt="" class="full" /></p>
<p><img src="https://i.imgur.com/5bZIdNp.png" alt="" class="full" /></p>
<p><strong>Step 2:</strong> <strong>Allow outbound traffic from VM to blob storage</strong> using port 449. I have used NSG rules in my VM where I denied internet access.</p>
<p><img src="https://i.imgur.com/AT4S4w8.png" alt="" class="full" /></p>
<p class="notice--success">🏆 <strong>Pro Tip</strong> <br />
<br />
Even though we assign public IP to VM and communicate to blob storage. All the traffic from VM to blob storage doesn’t go via internet. It goes via Azure Backbone since they are in the same region.</p>
<h2 id="why-we-need-virtual-network-service-endpoint">Why we need Virtual Network Service Endpoint?</h2>
<p>This above solution will work but what if you wanted to enable internet outbound from VM. It has security risk therefore, you route all traffic from VM to on-premise Firewall for inspection by enabling <a href="https://docs.microsoft.com/en-us/azure/firewall/forced-tunneling">Azure Firewall Forced Tunneling</a>.</p>
<p>This will un-necessary complexity to our network topology. Traffic has to hop multiple time before they reach to the destination.</p>
<p>Therefore, we have to think of different solution. <strong><a href="https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoints-overview">Service Endpoints</a></strong> is made for this kind of situation only.</p>
<p><img src="https://i.imgur.com/OIlmLEg.png" alt="" class="full" /></p>
<p>Traffic between the subnet and the storage account in the same region they reside in the Azure backbone network. Using Service Endpoint the VM inside subnet can connect to blob storage directly using optimized direct route.</p>
<h3 id="how-to-configure-service-endpoint-for-virtual-network">How to Configure Service Endpoint for Virtual Network?</h3>
<ul>
<li>
<p>Go to virtual network select Subnet where webserver is residing</p>
<p><img src="https://i.imgur.com/92M9sNh.png" alt="" class="full" /></p>
</li>
<li>
<p>Add a service endpoint for the Microsoft/Storage in the subnet of your Virtual Network.</p>
<p><img src="https://i.imgur.com/9eEIpsN.png" alt="" class="full" /></p>
</li>
<li>
<p>Now you can delete public IP address listed in blob storage account.</p>
<p><img src="https://i.imgur.com/YeL5c4P.png" alt="" class="full" /></p>
</li>
<li>
<p>In blob storage Firewall and Virtual network blade, add existing VNet and the subnet where you webserver resides in.</p>
<p><img src="https://i.imgur.com/Bq3yPZU.png" alt="" class="full" /></p>
</li>
</ul>
<h2 id="how-to-protect-data-ex-filtration">How to protect data ex-filtration?</h2>
<p>Service Endpoint allows VM to send data to any storage account living inside the same region. Therefore, it is quite possible that you can fetch prod data in VM which is in subscription-1 and send the data to DEV storage account which is in subscription-2. It is possible since both prod and dev storage accounts are in same region ( EAST US). This is security risk for your prod data. How do you protect this data ex-filtration?</p>
<p><img src="https://i.imgur.com/xkQjbgD.png" alt="" class="full" /></p>
<p>Well you need to add policy in Virtual Network Service Endpoint to restrict outside subscription blob storage account communication.</p>
<h2 id="service-endpoint-policies">Service Endpoint Policies</h2>
<p>Some malicious employee can send the data from the VM to some other secrete blob storage within same region. How do you prevent this?</p>
<p>You can limit the storage account that service endpoint has access to it by using <a href="https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoint-policies-overview">service endpoint policies</a>.</p>
<p><img src="https://i.imgur.com/SC6kWYt.png" alt="" class="full" /></p>
<p>You can go to service endpoint policies and create new policy, where you can restrict communication within your subscription resources only.</p>
<p><img src="https://i.imgur.com/HGvPY1j.png" alt="" class="full" /></p>
<p>Once you add policy and save it then it is fully secured.</p>
<p><img src="https://i.imgur.com/WCLZ10p.png" alt="" class="full" /></p>
<h2 id="allow-access-to-blob-storage-from-on-premise-or-home-pc">Allow access to Blob Storage from on-premise or home PC</h2>
<p>You can do NAT on on-premise and white list one public IP address in your storage account to allow access. However, if you do not want any public IP address access on your storage account then you must use <a href="https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-overview">Azure Private Endpoint</a>.</p>
<h2 id="references">References</h2>
<ul>
<li>https://www.youtube.com/watch?v=gxsitRRgylI&ab_channel=azuremonk-cloudinplainenglish</li>
</ul>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariDid you know By default azure storage accounts are accessible by the public internet. How can u restrict all public traffic in to your azure storage account and only allow your VNet resources to connect it. Lets learn the same in this article.Learn how to secure your Network2021-11-15T00:00:00+00:002021-11-15T00:00:00+00:00https://www.rupeshtiwari.com/network-security<blockquote>
<p>If you want to become Cloud Security Architect you must have basics of Networking. This article will discuss all fundamental concepts of networking. Learn about 6 security domains, understand the security is multi-pronged approach.</p>
</blockquote>
<h2 id="security-domains">Security+ Domains</h2>
<ul>
<li>Network Security</li>
<li>Compliance and Operational Security</li>
<li>Threats and Vulnerabilities</li>
<li>Application and Data and Host Security</li>
<li>Access Control and Identity Management</li>
<li>Cryptography</li>
</ul>
<h2 id="network-security">Network Security</h2>
<ul>
<li>Firewall, routers, switches</li>
<li>Protocol and Protocol Analyzer</li>
<li>ACLs, VLAN Management</li>
</ul>
<h2 id="compliance-and-operational-security">Compliance and Operational Security</h2>
<ul>
<li>Control Types</li>
<li>Risk Calculation</li>
<li>SLAs, Change and Incident Management</li>
</ul>
<h2 id="threats-and-vulnerabilities">Threats and Vulnerabilities</h2>
<ul>
<li>Malware and Viruses</li>
<li>Personally Identifiable Information (PII)</li>
<li>Attack Types - DDoS, Social Engineering</li>
</ul>
<h2 id="application-and-data-and-host-security">Application and Data and Host Security</h2>
<ul>
<li>Application Controls such as Secure Coding concepts, Cross Site Scripting</li>
<li>Device Security and Bring your own device (BYOD) concerns</li>
<li>Acceptable Use Policies</li>
</ul>
<h2 id="access-control-and-identity-management">Access Control and Identity Management</h2>
<ul>
<li>RADIUS, TACACS+ Kerberos, LDAP</li>
<li>Permissions, Authentication and Authorization</li>
<li>Tokens, Protocols and Methods</li>
</ul>
<h2 id="cryptography">Cryptography</h2>
<ul>
<li>Symmetric vs. Asymmetric</li>
<li>LAN, WAN, Wireless considerations</li>
<li>Steganography and other methods to hide/steal data</li>
</ul>
<h2 id="reference">Reference</h2>
<p>https://www.youtube.com/playlist?list=PL0bbor_qrUHgHF6cKAI8QNgkk4owMY7SF</p>
<hr />
<p><em>Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box.</em></p>
<div class="notice--success">
<strong>💖 Say 👋 to me!</strong>
<br />Rupesh Tiwari
<br />Founder of <a href="https://www.fullstackmaster.net">Fullstack Master </a>
<br />Email: <a href="mailto:rupesh.tiwari.info@gmail.com?subject=Hi">rupesh.tiwari.info@gmail.com</a>
<br />Website: <a href="https://www.rupeshtiwari.com">RupeshTiwari.com </a>
</div>
<p><img src="https://imgur.com/5fBatz9.png" alt="" /></p>Rupesh TiwariIf you want to become Cloud Security Architect you must have basics of Networking. This article will discuss all fundamental concepts of networking. Learn about 6 security domains, understand the security is multi-pronged approach.