Getting Started with Microsoft Azure Storage
www.Wintellect.com
Consulting/Training
About Wintellect
who we are
Wintellect is the only company that offers the combined value of world class consulting services along with
onsite, virtual and on-demand developer training. We help companies build better software, faster, helping you
maximize and protect your consulting and training investments through ongoing knowledge transfer.
consulting
training
Wintellect helps you build better software, faster,
tackling the tough projects and solving the software
and technology questions that help you transform
your business.
Wintellect's courses are written and taught by some of
the biggest and most respected names in the Microsoft
programming industry.




Architecture, Analysis and Design
Full lifecycle software development
Debugging and Performance tuning
Database design and development
 Learn from the best. Access the same training
Microsoft’s developers enjoy
 Real world knowledge and solutions on both current
and cutting edge technologies
 Flexibility in training options – onsite, virtual, on
demand
Consulting/Training
Consulting/Training
Agenda
 Azure Storage Services
 Blob Storage
 Table Storage
 Queues
 Files
 NoSQL
 DocumentDB
 MongoDB, RavenDB, CouchDB, etc
 HDInsight and HBase
 SQL Databases
 Azure SQL Database (PAAS)
 Installed SQL Databases (IAAS)
Consulting/Training
Azure Storage Services - Players
 Designed for availability and (massive) scalability
 Storage Types
 Blobs
 Tables
 Queues
 Files
 Consumption-based billing
Consulting/Training
Azure Storage Services - Benefits
 Cloud Storage - Anywhere and anytime access
 Blobs, Disks, Tables, Queues and Files
 Highly Durable, Available and Massively Scalable
 Easily build “Internet scale” applications
 More than 35 trillion stored objects
 3.5+ Million requests/sec on average
 Pay only for what you use
 Exposed via easy and open REST APIs
 Rich Client Libraries and Tools
Consulting/Training
Portals
Consulting/Training
Storage Types – Blob Storage
 Simple REST interface (Put, Get, Delete)
 Store binary objects – images, documents, media files, file storage
 Big Data – store raw data/logs and compute/map reduce over data
 Backups – data and device backups
 Blobs are stored in Containers, can simulate a file hierarchy by including the
“path” in the Blob name
 2 Types of Blobs - Block & Page Blobs.
 Majority of use is Block Blobs. Write set of blocks, then commit. Max size is 200GB.
 Page Blobs used for random r/w operations. Can be up to 1TB in size. Used for IaaS
VM images
Consulting/Training
Storage Types – Blob Storage
Consulting/Training
Blobs
Consulting/Training
Storage Types – Table Storage
 Non-relational key/attribute storage
 Tables store collections of “Entities”
 Up to 255 values per entity, up to 1MB in size per entity
 3 “reserved” values – PartitionKey, RowKey, Timestamp – you get 252
 Entities are specified by the compound key [PartitionKey & RowKey]
 The set of properties for which values are stored can vary from entity to
entity within the same table.
 Table content can be queried with a combination of operations
 (eq/ne) (gt/ge) (lt/le)
Consulting/Training
Storage Types – Table Storage
Consulting/Training
Tables
Consulting/Training
Storage Types – Queue Storage
 Reliable, decoupled messaging & asynchronous communication between apps
and/or components
 Decouple components/roles
 Web role to worker role communication
 Allows roles to scale independently
 Messages
 Can be up to 64 kb
 Messages live up to 7 days
 Not ordered, processed at least once
 Uses lease-based message retrieval
 A retrieved message is “invisible” until a period of time elapses or the message is deleted upon
completion
Consulting/Training
Storage Types – Queue Storage
Consulting/Training
Queues
Consulting/Training
Storage Types – File Storage
 Shared folder & file based storage
 Access content via REST API or as a standard SMB 2.1 file share
 SMB access allows File I/O APIs to be used
 SMB access is only within Azure Datacenter, only on VMs and Cloud Services
 Scenarios




“Lift and Shift” applications to the cloud that use on-premise files shares
Shared/Centralized application settings for distributed applications
Diagnostic share for logs, metrics, and crash dumps
Dev/Test/Debug
Consulting/Training
Storage Considerations - Billing
Cost
Description
Amount
Capacity
How much space are you using?
Varies on:
Storage type
Redundancy (LRS, ZRS, GRS, RA-GRS)
Transactions
How often are you trying to access it? $0.005 per 100,000 transactions (R or W)
Transfer / Egress
When does it leave the facility?
Consulting/Training
5GB / month free
$0.12 per GB
$0.19 per GB Asia Pacific East/West
Japan East/West
$0.25 per GB Brazil South
Storage Considerations – Choosing a Service
 Blobs vs Tables
 Blobs are cheaper to store and transfer (bytes vs UTF-8 strings)
 A blob can hold more data than an entity (200GB per block blob vs 1MB per entity)
 Blobs are more portable between vendors / less proprietary
 Tables are better at filtering, working with portions of the entity, or updating multiple
items (in the same partition) in a single transaction
 Queues
 How much data to put in the message vs the message as a reference to stored data
 Be sure to manage requests to an empty queue (pause/backoff) vs your SLAs
Consulting/Training
Azure Storage Queue vs Service Bus Queue
Storage Queue
Service Bus Queue
• Generally cheaper
• $.005/100,000 txns vs $.10/100,000 txns
• Note – Only storage has per month storage
charge
• Higher capacity (Storage account capacity vs 80GB)
• Lower latency (10ms vs 20-25 ms)
•
•
•
•
•
•
•
•
•
Long-polling
FIFO
Has closer on-premises analog
Supports pub/sub and routing
Duplicate detection
Message TTL > 7 days
Messages > 64 KB < 256 KB
Supports at-most-once processing
Supports local transactions
Consulting/Training
Additional Services, Tools and Libraries
 Azure Import/Export
 Move TBs of data into and out of Azure Blobs by shipping disk
 Submit and monitor jobs via REST and Portal
 All disks encrypted with BitLocker
 Client Libraries
 Simplify interaction with REST services
 Convenience capabilities – retry logic, parallelization, SAS tokens, …
 .NET (Desktop, Phone, Windows), Java, C++, Android, Node.JS, Python, PHP, Ruby, IOS (Coming Soon)
 Tools
 PowerShell commands
 CLI tools
 AzCopy 3.0 – Copy blobs and disks (Tables and Files in Preview Release)
Consulting/Training
NoSQL and Big Data
Consulting/Training
Why Use a NoSQL Technology on Azure?
To scale for lots of users and
lots of data
To work better with different
data formats, e.g., JSON
Pros: NoSQL technologies can
offer more scalability than
relational databases
Pros: Avoiding
object/relational mapping
makes code easier to write
Pros: NoSQL technologies don’t
have fixed schemas
Cons: Often lose some
benefits of relational
databases, e.g., database-wide
transactions
Cons: Limited BI tools;
persistent data designed
for a single application is
harder to share
Cons: Fixed schemas help
prevent errors
Consulting/Training
To work with data in a more
flexible way
Comparing Azure Managed Data Services
Category
Storage
Abstractions
Maximum
Database
Size
Stored
Procedures/
Triggers
Pricing
Query
Language
Transaction
Support
Secondary
Indexes
Yes
Written in
T-SQL
Units of
throughput
SQL
Database
Relational
Tables,
rows,
columns
DocumentDB
Document
store
Collections,
documents
100s of
TBs
Extended
subset of
SQL
All documents
in the same
collection
Yes
Written in
JavaScript
Units of
throughput
Table Storage
Key/value
store
Tables,
partitions,
entities
100s of
TBs
Subset of
OData
queries
All entities in
the same
partition
No
None
GBs of
storage
500 GB
SQL
All rows and
tables in a
database
Consulting/Training
DocumentDB CRUD Operations
Consulting/Training
Other NoSQL Azure Options
 Utilize via Marketplace, Hosted Service, or VM
 MongoDB
 Two options via marketplace
 RavenDB
 Just added to the marketplace
 CouchDB
 https://cloudant.com/sign-up/
 Azure storage location in beta
 All of the above can also be installed via VMs
Consulting/Training
Adding NoSQL services from Azure marketplace
Consulting/Training
Data Today - More and more and more and . . .
Consulting/Training
Big Data Analytics - The industry standard: Hadoop
Hadoop
MapReduce
...
HBase
YARN
Consulting/Training
Azure HDInsight
provides these as a
managed service
HDInsight MapReduce
An illustration
HDInsight MapReduce
Excel
Hive
Pig
...
MapReduce
Job
VM
VM
VM
Logic
Logic
Logic
HDFS API
HDInsight HBase is also
implemented on this API
and relies on Azure Blobs
1000110100110
0111101111101
1011010001101
1000110100110
0111101111101
1011010001101
1000110100110
0111101111101
1011010001101
Blob
Blob
Blob
Azure Blobs
Consulting/Training
Relational Databases
Consulting/Training
Database Options
 Azure SQL Database
 Database in a VM
 SQL Server, Oracle, MySQL (VM cost includes product license cost)
 Store data using attached disks in Blob Storage
 Access an on-premises database from Azure
 Using VPN or BizTalk Services Hybrid Connection (firewall-friendly, secure
connection from Azure to an on-premises TCP port, currently for Web Sites &
Mobile Services only)
Consulting/Training
Azure SQL Database
 SQL Server Relational DB as a service
 Same SQL Tabular Data Stream (TDS) interface
 Simple provisioning
 Security
 SQL Server authentication only
 Firewall – accessing IP addresses must be whitelisted (intra-Azure access can be
enabled/disabled)
 Azure Table Storage & Azure SQL Database - Compared & Contrasted
- http://msdn.microsoft.com/en-us/library/jj553018.aspx
Consulting/Training
Azure SQL Database vs “Installed” SQL Server
 Unsupported features:
 Configuration is managed
 Distributed transactions, distributed
 3 high-availability replicas are
views
maintained for every database
 All tables require a clustered index
 Maximum DB size is 500GB
 Does not include other SQL Server
Services (Reporting, SSIS, etc.)
 Subset of Dynamic Management
Views
 SQLCLR (CLR Data Types are supported)
 SQL Agent, SQL Profiler
 Native encryption (encryption at
application level)
 Integrated Windows Authentication
(SQL authentication only)
 http://msdn.microsoft.com/library/azure
Consulting/Training
/ff394115.aspx
Development & Tooling
 Languages
 .NET via ADO.NET
 OData, EF, NHibernate, etc
 JavaScript with node-sqlserver
module
 C/C++ via ODBC
 Java via MSFT JDBC
 Tools
 SQL Database Management Portal
 Web-based tooling for working with tables,
views, stored procs, query editor, etc
 SQL Server Data Tools
 Integrate into VS IDE
 SQL Server Management Studio (2008
R2+)
 Others
 SQL Server Command-Line utilities
 PHP via MSFT PHP
 CA ERWIN
 DBArtisan
Consulting/Training
Choosing Tiers & Levels
Service
Tier
Performance
Level
Common App
Pattern
Performance
Business Continuity
Max DB
Size
Transaction Perf. DTUs
Objective
PITR
DR / GEO-Rep
Basic
Basic
Small DB
2 GB
Reliability / Hr.
Past 7
Days
DB Copy +
Manual Export
Standard
S1 / S2
Wrkgp/cloud app,
multiple concurrent
operations
250 GB
Reliability / Min. 15/ 50 Past 14
Days
DB Copy +
Manual Export
Premium
P1 / P2 / P3
Mission Critical, High 500 GB
volume, Many
concurrent Users
Reliability / sec.
Active Georeplication
Consulting/Training
5
100/
200/
800
Past 35
Days
SQL Azure
Consulting/Training
Resources
Guidance:
Tools:
 Microsoft Azure Storage – What’s New, Best Practices
 Storage Explorers

List as of 5/15/2014 - http://j.mp/AzureStorageExplorers
 Jeffrey Richter’s Guide to Working with Azure
Storage Tables via C# and other .NET Languages
and Patterns (Build 2014) http://channel9.msdn.com/Events/Build/2014/3-628
 Patterns & Practices for sharing Azure SQL Database
(plus helper library):
http://channel9.msdn.com/Shows/DataExposed/SqlDbShardingIntro
 Documentation – http://wintellect.com/guide-toworking-with-azure-storage-tables
 NuGet Package -
Demos/Sample Code:
http://www.nuget.org/packages/Wintellect.Azure.St
orage/
 Azure SAS Generator App for Windows

http://j.mp/RichterAzureSASGenerator
 Microsoft Azure Code Samples –
https://code.msdn.microsoft.com/windowsazure/
 Patterns & Practices for sharing Azure SQL Database
(plus helper library):
http://channel9.msdn.com/Shows/DataExposed/SqlDbShardingIntro
Consulting/Training
Demos/Samples Used in Presentation
 Download and upload files to Azure Storage from Universal Windows Apps
 https://code.msdn.microsoft.com/windowsazure/Universal-Windows-Apps-1ea96689
 Visual Studio Azure Storage Queues Quickstart
 http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-queues/
 Azure DocumentDB Code Samples
 https://code.msdn.microsoft.com/windowsazure/Azure-DocumentDB-NET-Code-6b3da8af
Consulting/Training