Disruptive Technologies for 2015: Software-defined Storage, Virtual SAN, Flash and Hybrid Cloud Predictions

From StorageNewsletter

George Teixeira

President and CEO, DataCore Software

Major storage hardware vendors saw flat growth or significant sales declines as a result of all the disruptive forces hitting the data storage market this year. Virtual SANs, hybrid clouds, flash for performance and software-defined storage have become the rage but enterprises are still struggling to implement these new technologies in a practical manner.  

Below are four related enterprise software predictions for 2015. 

#1: Software-defined Storage Will Go Mainstream in 2015

The value proposition for Software-defined storage is inevitable. It will continue to disrupt enterprise storage in 2015. It will further commoditize underlying storage devices and raises storage features and services from being locked to those devices. True software-defined storage platforms like DataCore will deliver comprehensive storage services that allow devices to achieve a new level of productivity by being able to work cross-platform and infrastructure-wide.  Bottom-line, the compelling economic benefits, better productivity and the need for greater agility to meet future requirements will drive software-defined storage to become a mainstream solution in 2015.

#2: Hybrid Clouds will become Practical in 2015

Enterprises are dealing with both on-premise storage and off-site cloud storage (hybrid cloud). This will become a bigger issue as customers become smarter about what cloud workloads are practical. On-premise storage is usually allocated for active data such as transaction-oriented business. The cloud will still be primarily used for cold data, back-up and disaster recovery due to internet speeds. New solutions are emerging such as DataCore and Microsoft StorSimple, which combine to allow data (from any storage) to be seamlessly migrated from on-premise to a cloud such as Microsoft Azure. This will fuel the larger trend, which is for enterprises to do a mix of on-premise and cloud. In addition, while doing disaster recovery from the cloud remains complex, new integration tools and more automated processes are on the way to make this a more practical solution.

#3: Disk and flash software stacks must span both worlds. Flash is excellent for specialized workloads that require high speed reads such as databases, but it is not a cost-effective solution for all workloads and still makes up a very small fraction of the installed storage base overall. On the other side of the spectrum are low-cost SATA disk drives that continue to advance and use new technologies like helium to support huge capacities, up to 10 TB per drive, but they are not highly performant and are slow.  Write-heavy transaction workloads also need to be addressed differently. (See New Breakthrough Random Write Acceleration and Impact on Disk Drives and Flash technologies). All flash is still not practical due to the costs involved, and the large installed base of disk storage that must be addressed. Comprehensive software stacks  will rise in importance in 2015, since users must be able to optimize the cost and performance trade-offs and migrate workloads to the right resources needed whether flash or disk. Software-defined storage done right can help unify the new world of flash with the existing and still-evolving world of disks. Both have a future.

#4: Servers as Hyper-converged Virtual SANs will Continue to Displace Traditional Storage Arrays

The latest generation of servers are powerful and will continue to support even larger amounts of storage. Software defined storage software such as DataCore’s Virtual SAN software will further drive servers to be transformed into powerful storage systems and, in effect, enterprise-class virtual SANs.

See the Dell 4Enterprise Blog post, “Dell PowerEdge Servers Make Great Software-Defined Storage Solutions,”  and the  “Software-Defined Virtual SAN Solutions Powered by FUJITSU PRIMERGY Servers” as examples of this trend. 

Happy Holidays!

DataCore's New Random Write Accelerator for Databases,SQL, ERP, OLTP and RAID-5 workloads

DataCore in its most recent release has added even more powerful and comprehensive enterprise-class functionality to its SANsymphony™-V10 storage services platform and its ‘any hypervisor, any storage’ DataCore™ Virtual SAN. The newest enhancements further boost I/O performance and double the scale of hyper-converged storage systems to 64 nodes.  

Accelerate Performance Up to 30X for Random Write Intensive Applications such as OLTP

The new software release includes a new Random Write Accelerator, which yields up to 30 times faster performance for random-write-heavy workloads that frequently update databases, ERP and OLTP systems. Even greater performance gains can be realized on RAID-5 protected datasets that spread data and reconstruction information to multiple locations across different disk drives.

To Learn more, see our webinar series and demo:

SSD-like Speeds from Spinning Disks; Faster Response and Reduced Write-wear for Flash
The remarkable write performance speed-ups are most pronounced when using low-cost spinning disks (e.g., SATA disk drives) where each write incurs significant rotational and seek delays to mechanically update blocks on spinning platters. Higher speed flash-based technologies (SSDs), especially prone to write-amplification and rapid wear, benefit from the Random Write Accelerator as well, delivering up to 3.6 times more IOPS.

Double the Scalability – Hyper-Converged Systems up to 64 Nodes and Over 100 Million IOPsThe new version of DataCore Virtual SAN supports larger scale configurations that can be managed centrally and organized to match business operational needs. With these latest enhancements, customers can now spread their workloads over more servers in a cluster and share resources across multiple clusters in order to improve their response and throughput. 

Doubling the possible number of nodes in a DataCore Virtual SAN is particularly valuable for latency-sensitive applications distributed over large-scale clusters, as well as virtual desktop (VDI) deployments for thousands of users.

Scaling out the configuration across more nodes enables better distribution of the workloads and segmentation of the physical storage capacity. DataCore’s zero-touch failover capabilities ensure workloads and applications are not disrupted in the event of a node or storage failure. The internode mirrors can also be stretched over metro-wide distances up to 60 miles (100 kilometers) to improve overall availability and business continuity. Scaling out to more nodes also enhances the overall resiliency of the infrastructure.

Press Release: 

DataCore Introduces Major Breakthroughs in Performance, Scalability and Quality of Service in its New Software-Defined Storage and Virtual SAN Release

SearchVirtualStorage: DataCore boosts performance, adds Storage Accelerator and QoS to SANsymphony-V10 and Virtual SAN

DataCore Software's SANsymphony-V10 now features a random write accelerator to improve performance of write-intensive workloads.

DataCore Software Corp. updated its SANsymphony-V10 storage virtualization software to boost performance and scalability, add quality of service capabilities and provide tighter integration with Microsoft Azure cloud services.

...The DataCore software runs on standard x86 servers and manages the capacity of their internal disk and/or solid-state drives (SSDs) as a shared storage pool.

One of the key features in the SANsymphony-V10 PSP1 update is the random write accelerator. DataCore said the new technology converts random writes into sequential writes to improve the performance of write-intensive workloads such as databases, ERP and online transaction processing systems, and workloads that use RAID 5 data protection.

Jon Toigo, CEO and managing principal at Toigo Partners International LLC, said DataCore could also position the random write accelerator to address the performance-degrading I/O blender phenomenon. The I/O blender effect occurs when multiple virtual machines try to write data at the same time.

"It's basically doing some log-file structuring and some write coalescence that turns a whole bunch of random writes into sequential writes so that you speed up the performance of your storage," said Toigo, who is a user of DataCore software.

Updates boost performance, management

Augie Gonzalez, director of product marketing at DataCore, said Iometer testing of 100% random write workloads with a 4K block size showed 33 times faster performance, with low-cost SATA hard disk drives (HDDs) running on Dell servers. According to Gonzalez, using identical test conditions, the random-write-accelerated SATA HDDs outperformed multi-level cell (MLC) SSDs that did not use the new "turbo charger" feature.

Gonzalez said, when using the random write accelerator, the SSDs also saw performance 3.6 times faster than without the write accelerator. The acceleration technology, coupled with the product's caching mechanism, also helps to reduce the write-induced wear-out factor of SSDs, he said.

SANsymphony-V10 PSP1 increases the maximum number of server nodes supported in hyper-converged configurations from 32 to 64. That allows customers to spread workloads across more servers in a cluster to boost throughput. The company said each server can support more than 1 million IOPS, and the extra scalability can be especially helpful with distributed latency-sensitive applications and large virtual desktop infrastructure (VDI) deployments.

"Let's say the production cluster is running out of space," Gonzalez said. "Before this customer can stand up another server and put more storage on it, they've got to get over this afternoon's processing window. Their test and dev cluster has extra space. Normally those two couldn't touch each other. In the case of DataCore, having a common storage pool, we can basically loan that production cluster some temporary capacity."

SANsymphony-V10 PSP1 also introduces quality of service (QoS) controls to enable customers to manage and regulate the system to ensure that low-priority workloads don't monopolize resources that more critical applications need. The new release features host group access control to let administrators assign different application hosts to different segments of the storage pool.

"The combination allows you to segregate local partitions of the data to where you need it but also to remap it when you need to," DataCore CEOGeorge Teixeira said.

Gonzalez noted that DataCore's QoS applies not only to capacity but also to Fibre Channel and iSCSI connections with the host. With PSP1, the company has added utilization tracking capabilities and chargeback reports to enable customers to manage their storage infrastructure as a private cloud.

Another new option in SANsymphony-V10 is the ability to have a standby node to enable full performance during software updates, equipment refreshes and maintenance tasks or to offload responsibilities from an overtaxed node. The product already supported automatic failover between nodes in the event of a storage or device failure, but there was a performance impact. DataCore customers will now have the option to shift workloads to a standby node without disrupting the applications or degrading performance and throughput.

Teixeira said the standby node will be especially helpful for customers such as banks or hospitals that use metro clusters or metro mirroring stretching across different locations. While this type of feature traditionally appeals to customers with no tolerance for failure, and who are willing to pay for an extra server to get an even higher level of availability, falling server prices could bring it to more midmarket IT organizations.

DataCore looks to public cloud for backup and DR

With the PSP1 update, DataCore also gives customers additional options to send data to public cloud storage for off-site backups, archives and disaster recovery. The company tightly integrated and certified its SANsymphony-V10 to work with on-premises Microsoft StorSimple hybrid cloud arrays, which can serve as a bridge to Microsoft's Azure public cloud storage.

"One of the advantages we bring to Microsoft is that their array only works with iSCSI storage," Teixeira said. "We work with both Fibre Channel and iSCSI, so we can virtualize all the storage on-premises, whether it is Fibre Channel or iSCSI based, and we can automatically tier and migrate it to the StorSimple gateway."

...Teixeira said he is especially focused on Windows Azure because the service caters to business use involving databases such as Microsoft SQL Server and Exchange Server.

"When we talk to our customers, what they're looking for is a rock-solid commercial capability out there, and Windows Azure seems to be that platform for most of them," Teixeira said. "So, getting it jointly certified was key for us. I think it's probably where we're going to see the most pickup from our customer base."

See Full Article at: http://searchvirtualstorage.techtarget.com/news/2240234858/DataCore-boosts-performance-adds-QoS-to-SANsymphony-V10 

ComputerWorld: House of Travel Australia finds value in DataCore Software-defined storage

Travel agent firm can scale its storage environment up or down when required

House of Travel Australia has used a software-defined storage services platform to reduce capital expenditure and improve the resilience of its storage environment.

The company has a network of 460 personal travel managers based around Australia.

House of Travel Australia's director of IT, Matthew Harris, said it implemented DataCore’s SANsymphony-V software in May 2014 to manage virtualized storage for its VMware ESX systems.

“The expenditure on this software solution saved significant capital expenditure while providing a stable operational environment,” he said.

“The biggest benefit is peace of mind for our future. We can scale [storage] up or down and scale to multiple sites. DataCore can handle any level of workload that I need to give to it."

House of Travel operates two business units, which are TravelManagers and HOOT Holidays. Both business units store their data on the DataCore solution.

Aside from giving Harris peace of mind, the system proved its worth when there was a hardware failure with one of its IBM servers earlier in 2014.

“The beauty of the DataCore system is that it keeps all of my data on two different servers. The [IBM] server went totally dead and there was no chance to do anything,” he said.

“DataCore indicated that the data was on another server and made it available to the VMware ESX system. As far as the VMware system was aware, it lost some processing and random access memory [RAM] power but all of its storage was still there. It automatically re-started our virtual machine on to the other hardware storage. The business suffered about one minute of sluggish performance but there was no downtime or data loss,” said Harris.

He is now looking at using DataCore’s Asynchronous offsite replication. This will provide an exact copy of House of Travel Australia’s data offsite so the firm can recover information quickly in an emergency. Harris plans to implement the software in Q1 of 2015.

More at: http://www.computerworld.com.au/article/560844/house-travel-australia-finds-value-software-defined-storage/

The Register: DataCore lifts baton, strikes up the 64th SANsymphony with latest update release for SANsymphony-V and Virtual SAN

http://www.theregister.co.uk/2014/11/27/datacore_drives_sansymphony_64_ways_twice/

Comment DataCore has updated its SANsymphony and Virtual SAN software products to reach 100 million IOPS and 64PB capacity. It's also getting into bed with Azure.

SANsymphony-V10 PSP1 builds a SAN from a cluster of servers, aggregating each server's storage into a central virtual pool. The company has built its product on virtual SAN technology which VMware has now blessed as a concept with its VSAN technology...

...DataCore says that - unlike VMware's ESXi-only VSAN - its Virtual SAN supports any hypervisor and any storage.

Its software enhancements mean the following things:

• Systems can scale out to 64 nodes, up from 32
• They can scale up to 64PB configurations
• Notionally they can deliver more than 100 million IOPS

Claiming a 64PB and 100 million IOPS capability is bold; achieving it will be impressive.

The SANsymphony V10 PSP1 product has quality of service (QoS) features providing policies which manage access to storage tiers divided between on-premise and public clouds, defining storage resource allocation and providing Service Level Agreement capability. This ensures high priority applications aren't slowed by less important ones whose consumption of resources is controlled by capping their data transfer rate and IOPS.

DataCore says storage resources can be logically segregated, monitored and regulated on a departmental basis. This SW release also adds utilisation tracking capabilities and chargeback reports.

Random Write Accelerator technology has been added, which - DataCore says - benefits ERP, OLTP and database applications with up to 30 times faster writes. This secret sauce is said to provide a greater than 30x random write improvement on RAID-5 protected datasets with data and metadata - reconstruction information - spread across multiple locations on different disk drives.

It is also said to be especially effective with slower SATA disk drives exhibiting longer rotational and seek delays when updating data blocks. And flash can benefit too, with SSDs having lower write amplification and less wear, as well as a claimed 3.6x increase in IOPS.

This sound to us like some kind of random write buffering transforming many random writes into fewer sequential ones.

Doubling the node count in Virtual SANs helps increase the number of applications they support or the number of users in the VDI use case. DataCore says resources can be shared across multiple clusters as well.

There is zero-touch failover to provide non-disruptive failover if either a storage or server component fails. Internet mirrors can span 60-mile (100 kilometre) distances.

Microsoft Azure integration

SANsymphony-V can use Microsoft's Azure cloud core via a Microsoft StorSimple array with a very, very long name - the DataCore-ready certified Azure StorSimple Hybrid Cloud Storage Array.

In a hybrid cloud scheme Azure provides backend geo-redundant cloud storage, with SANsmphony-V auto-tiering on-premise storage to StorSimple Arrays and hence Azure. The StorSimple arrays can be pooled like any other SanSymphony storage.

Working set backups (snapshot copies) can be sent to Azure through the StorSimple gateways. They can be retrieved from separate locations by other StorSimple arrays or by Azure-hosted applications.

If accessing hosts use Fibre Channel, SANsymphony-V converts this protocol to iSCSI.

Bits and bobs

There's more; SANSymphony V10 PSP1 gets performance-monitoring capabilities, wizards to automate processes such as software updates, and enhanced CDP capabilities for restoring from known good points-in-time.

There is now a standby node capability whereby workloads can be redistributed when an existing node is taken out of service or gets overburdened in a SANsymphony-V group.

The latest Fujitsu PRIMERGY BX and CX servers are certified as DataCore Ready Software-defined Storage servers. Fujitsu's ETERNUS DX arrays have gained DataCore Ready Certification.

SANsymphony V10 PSP1 and VirtualSAN are shipping now with the enhancements above. DataCore's 10,000 customer sites should be pleased with this raft of new features as their mature product gets significantly extended. 

DataCore Offers Pre-Tested Software-defined Storage Solutions Based on Dell PowerEdge Servers and is Named a Certified Dell Technology Partner

http://www.it-director.com/channels/reseller/news_release.php?rel=44665

DataCore, a leader in software-defined storage and converged virtual SAN solutions, is now a certified Dell Technology Partner. The program certification recognises DataCore for successfully passing the rigorous testing process required to validate the integration and interoperability of its software on Dell PowerEdge server platforms. The partnership instills customer confidence and gives enterprise IT greater flexibility and more freedom to choose certified, pre-tested platforms and infrastructures that combine industry-leading Dell PowerEdge servers and storage with DataCore’s software-defined storage software solutions.

DataCore SANsymphony™-V and Virtual SAN software transform Dell PowerEdge servers into fully featured software-defined storage servers that are able to virtualise, enhance and derive the utmost productivity from a company’s present and future storage investments. DataCore’s powerful software solutions enable Dell’s PowerEdge servers and the full range of Dell’s storage products to be easily integrated with all the popular brands and models of installed storage and solid state disk technologies regardless of the original vendor. With DataCore software, it is easy to leverage Dell PowerEdge performance acceleration over the entire infrastructure of storage assets. The combination supports powerful features like metro-wide shared storage for clusters and business continuance, and automates tiering, provisioning and migration of data storage across the diversity of new or installed disk and flash-based technologies.

Dell and DataCore solutions combine to deliver enterprise-class performance and highest levels of availability. In addition, Dell PowerEdge servers and DataCore Virtual SAN software make an ideal solution for the mid-market, especially for Microsoft virtualisation projects and mixed Hyper-V and VMware environments running business critical applications such as SQL, SharePoint, Exchange, Dynamics, SAP, Oracle and VDI. In these use cases, DataCore users report nearly a 100 percent reduction in storage-related downtime, storage costs decreased by up to 75 percent and I/O performance improvements of virtualized applications by up to 10x.

DataCore is showcasing its DataCore Ready Software-Defined Storage (SDS) software  running on Dell PowerEdge servers at the Dell World 2014 conference this week in Austin, Texas. Attendees, please stop by DataCore’s Booth, #3E20A, to meet the DataCore team and learn more about the newest DataCore and Dell solutions.

For more information on the DataCore and Dell partnership, please see:

Partner Page: DataCore and Dell Joint Solutions

Dell4Enterprise Blog Post: Dell PowerEdge Servers Make Great Software-Defined Storage Solutions

Data Sheet: DataCore Software-defined Storage on Dell PowerEdge Servers

eWeek: DataCore Ready Software-Defined Storage Servers Running on Dell PowerEdge Debut

DataCore’s SANsymphony-V software works with Dell’s PowerEdge servers and a range of Dell Storage MD Series, SC Series and PS Series arrays.

Software-defined storage specialist DataCore announced the launch of its Ready Software-Defined Storage (SDS) software, running on Dell PowerEdge servers. 

See the full Post on Dell's web site: Dell PowerEdge Servers Make Great Software-Defined Storage Solutions

See the new DataSheet: DataCore Software-Defined Storage on Dell PowerEdge Servers

DataCore’s SANsymphony-V software works with Dell’s PowerEdge servers and a range of Dell Storage MD Series, SC Series and PS Series arrays. 

The DataCore software makes it possible to add PowerEdge performance acceleration to the pool of heterogeneous storage systems, support features such as metrowide stretch mirroring for business continuance, as well as simplified migration and workload mobility from traditional storage systems to new platforms. 

DataCore software solutions enable once-isolated storage systems--from flash and disks located within servers, to external SAN arrays and public cloud storage--to become part of an enterprisewide accessible virtual pool, classified into tiers according to their unique characteristics. 

Different brands of storage, standalone converged systems and hypervisor-dependent Virtual SANs and external storage systems no longer need to exist as islands.

Instead, they can be integrated within an overall storage infrastructure. The system administrator can provision capacity, maximize utilization and set high-level policies to allow the software to select the most appropriate storage tier and paths to achieve the desired levels of performance and availability.

"Enterprises are experiencing significant challenges associated with storage, including increasing pressure to maintain uninterrupted service, the need to scale with increasing capacity demands, and more onerous IT data retention policies," George Teixeira, president and CEO of DataCore, told eWEEK. "In addition, the cost and performance of where data is stored matters, especially when we consider what storage we put in the cloud, and how we use more expensive flash storage–this all adds to the growing complexity. To top it all off, IT managers need to work within ever-diminishing IT budgets."

...The company’s Virtual SAN also works with all major hypervisors (including VMware vSphere and Microsoft Hyper-V) and many popular storage offerings, Teixeira noted.

"While technical alternatives exist from competitors, they often prove prohibitively expensive except for the most affluent of IT organizations, and suffer from large recurring costs," he said. "DataCore offers affordable solutions that take full advantages of resources already in use. Furthermore, the advanced software services endure as underlying storage hardware is replaced, to eliminate the wasteful rip-and-replace processes which drive up costs." 

See the full article at: http://www.eweek.com/small-business/datacore-ready-software-defined-storage-servers-debut.html#sthash.gpATu72R.dpuf 

Get better involved with DataCore's Channel!

Don’t be left behind, genuine Software-defined Storage is the future and you need to know how best you can leverage DataCore Software’s Channel model. Join us at our Channel Event and get the most out of DataCore’s Channel opportunities.

DataCore Software is hosting a Channel Event in the RACV Melbourne on the 12^th November 2014 at 5pm-7.30pm. This is your opportunity to find out more about DataCore’s Channel and how to best reap the benefits.

This Channel Event is an exclusive opportunity to network with some of your colleagues, enjoy delicious canapés and drinks, along with an interactive Panel Discussion. The Panel Discussion will be short, sweet and to the point giving you an overview of DataCore’s Channel model to date, why customers choose DataCore, while also educating you on the importance of being fully engaged and involved with Software-defined Storage.

DataCore can fundamentally solve your customer’s most pressing storage-related challenges, while also changing the economics of managing their data storage. They want to know about DataCore!


On the Panel with be Peter Thompson, VP APAC DataCore, David Bull, Director of Technical Sales at DataCore and one of DataCore’s esteemed end-users, Mark Naidoo, the IT Infrastructure Manager at Transdev.

Mark Naidoo will be on the Panel to speak about why Transdev chose DataCore Software to drastically improve and enhance their storage environment. He will also share what DataCore has enabled Transdev to do so far. You will get the opportunity to ask Mark questions about his choice of DataCore Software, and his experience with the Channel. 

This is your time to find out why organisations around the globe are turning to DataCore for their storage.

We ask you to please register your attendance to our Channel Event and we look forward to seeing you on the 12^th November 2014. If you would like to bring along a colleague(s) please get them to register also.

Webinar: Don’t hesitate any longer. The future of data storage is already here.

Let us help you solve your most pressing storage-related challenges and fundamentally change the economics of managing your data storage. Seamlessly scale up and scale out choosing the most appropriate hardware manufacturer for each class of storage and servers.

Register here for this webinar on Monday 10th November at 2pm-3pm (AEST).

SANsymphony™-V10 our flagship 10th generation storage virtualization solution. In use at over 10,000+ customer sites, 25,000+ installations; maximising IT infrastructure performance, availability and utilisation by virtualizing storage hardware.

Join David Bull, Director Technical Sales on Monday 10^th November to find out how DataCore Software is “empowering businesses with an enduring and dynamic ‘Software-driven Storage’ architecture, liberating storage from static hardware-based limitations”.

DataCore at Dell World Unveils Software-Defined Storage Powered by Dell PowerEdge Servers

Combination to deliver storage flexibility, performance across enterprises

DataCore is showcasing its DataCore Ready Software-Defined Storage (SDS) software  running on Dell PowerEdge servers at the Dell World conference in Austin, Texas.

DataCore’s SANsymphony-V is a comprehensive and scalable storage services platform designed to maximize the performance, availability and utilization of IT assets.

Different brands of storage, standalone converged systems and hypervisor dependent Virtual SANs and external storage systems no longer need to exist as ‘islands’ — with DataCore, they can be integrated within an overall storage infrastructure.

DataCore’s Virtual SAN solution enables enterprises to virtualize and empower locally converged systems as well as go beyond other approaches in the market by providing a growth path to incorporate external storage and avoid the sprawl of storage islands.

“The combination of Dell’s advanced PowerEdge servers and our proven software-defined storage platform can empower storage to whole new price performance levels,” said Steve Houck, COO of DataCore. “While we work with all the popular hypervisors, the price performance of Dell PowerEdge servers lets us target a largely unfulfilled segment of the marketplace – that is, fast, affordable and simple-to-use Virtual SANs that support Microsoft virtualization and Hyper-V projects and mixed VMware environments running critical applications such as SQL, SharePoint, Exchange, SAP, Oracle and VDI.

DataCore’s users report nearly a 100 percent reduction in storage-related downtime, storage costs decreased by up to 75 percent and I/O performance improvements of virtualized applications by up to 10x.

The newest enhancements to DataCore’s Virtual SAN, available this month, will further boost I/O performance and enterprise class scaling, doubling the amount of nodes supported up to 64 and providing the capability to deploy larger-scale 64 petabyte configurations with the power to drive over 100 million IOPs.

http://www.ciol.com/datacore-unveils-software-defined-storage-servers/ 

DataCore’s Answer to Random Write Workloads: Sequential Storage

by Jeffrey Slapp Technical Product Specialist / Systems Engineer

Introduction
DataCore Software has developed another exciting new feature extending the arsenal of enterprise features already present within SANsymphony-V. This new feature serves to enhance the performance of random write workloads which are among the most costly operations that can be performed against a storage system. The new Sequential Storage feature will be available in SANsymphony™-V10 PSP1 scheduled for release this month.

Internal testing with the Sequential Storage feature and 100% random write workloads yielded significant performance improvements for spinning disks (>30x improvement) and even noteworthy improvements for SSDs (>3x improvement) under these conditions. The specific performance numbers will be covered later in this article.

The actual performance benefits will vary greatly depending on the percentage of random writes that make up the application’s I/O profile and the types of storage devices participating within the storage pool. Additionally, the feature is enabled on a per-virtual disk basis, allowing you to be very selective about when to apply the optimization.

Basis For Development
As applications drive storage system I/O, DataCore’s high-speed caching engine improves virtual disk read performance. The cache also improves write performance, but its flexibility is limited due to the need to destage data to persistent storage. In many environments the need to synchronize write I/O with back-end storage becomes the limiting factor to the performance that can be realized at the application level; hence the purpose of this development.

With certain types of storage devices, there are significant performance limitations associated with non-sequential writes compared with sequential writes. These limitations occur due to:

• Physical head movement across the surface of the rotating disk
• RAID-5 reads to calculate parity data
• Write amplification inherent to Flash and SSD devices

DataCore SANsymphony-V software presents an abstraction to the application — a virtual SCSI disk. The way that SANsymphony-V stores the data associated with these virtual disks is an implementation detail hidden from the application. Data may be placed invisibly across storage devices in different tiers to take advantage of their distinct price/performance/capacity characteristics. The data may also be mirrored between devices in separate locations to safeguard against equipment and site failures. The SANsymphony-V software can use different ways to store application data to mitigate the aforementioned limitations, while not changing the abstraction presented to the applications.

Functional Details
Sequential Storage changes the way SANsymphony-V stores data written to the virtual disks by:

• Storing all writes sequentially
• Coalescing writes to reduce the number of I/Os to back-end storage
• Indexing the sequential structure to identify the latest data for any given logical block address
• Directing reads to the latest data for a block using this index
• Compacting data by copying it and removing blocks that have been rewritten

Performance Details
Now the part everyone is waiting for – the performance numbers. There are three main states to consider from a performance perspective:

• Base – the underlying level of performance that can be achieved with a 100% random write workload, without Sequential Storage enabled.
• Maximum – the performance that can be achieved with a 100% random write workload, with Sequential Storage enabled but without compaction active.
• Sustained – the performance that can be sustained with a 100% random write workload, with Sequential Storage enabled and with compaction active.

The greatest performance is achieved during the Maximum state. When the virtual disk is idle, a background level of compaction will occur to prepare the system to absorb another burst of random write activity. That is, the background compaction will prepare the virtual disks to deliver performance associated with the Maximum state.

The following performance has been observed using IOmeter running a 100% write, 100% random workload with a 4K block size and 64 outstanding I/Os:

	Base IOPS	Maximum IOPS	Sustained IOPS
Linear 20 GB volume, SATA WDC 1 TB drive	327	19,500	11,000
Linear 20 GB volume, SSD 840 EVO 250 GB Pool	10,000	62,000	36,000
Mirrored 100 GB volume, PERC H-800 RAID-5 Pool	860	67,000	40,000

Interesting Observations
The above results highlight 3 key observations:

• Significant acceleration (>30x improvement) of low-cost SATA disks for random write loads is possible. In fact in this particular test with DataCore, the resulting sustained performance of 11,000 IOPS actually exceeded that of a conventional Solid State Disk which ran at 10,000 IOPS.
• The Solid State Disk also displayed improved performance going from 10,000 IOPS to 36,000 IOPS (>3x improvement).
• Write intensive RAID-5 workloads displayed the greatest amount of improvement from 860 IOPS to 40,000 IOPS (>45x improvement).

Conclusion
DataCore’s Sequential Storage capability aims to address a limitation every storage system experiences to some extent. Random writes not only severely impact application performance within mechanical systems such as magnetic disks, they can also drastically reduce the performance and shorten the lifespan of SSD/flash based devices because of the write amplification effects produced from the write I/O pattern (see this publication for more detail). You can expect this feature along with many others in SANsymphony™-V10 PSP1 due out later this month.

New Hybrid Cloud Software-Defined Storage Solution Certified; Combines DataCore Storage Virtualisation flexibility with Microsoft Azure StorSimple

Hybrid Cloud Storage Solution Enables Enterprises to Leverage On-premise, Private Cloud and Public Cloud Resources to Optimize Storage Performance and Costs

DataCore today announced at the Powering the Cloud conference that the newest Microsoft Azure StorSimple 8000 and the StorSimple 7000 Series Hybrid Cloud Storage Arrays are certified as DataCore Ready. StorSimple and Microsoft Azure combine with DataCore’s Virtual SAN andSANsymphony-V storage virtualization and services platform to offer a hybrid cloud storage solution that enhances the utilization and productivity of existing primary storage residing locally, while providing a cost-effective way to utilize and integrate off-premise cloud storage for backup, archive and disaster recovery. The StorSimple Hybrid Storage Arrays were awarded the DataCore Ready certification after the successful completion of a rigorous verification and test process.

DataCore and Microsoft Combine to Tier, Migrate, Backup, Archive and Do Recovery from the Cloud  
StorSimple hybrid cloud storage appliances are able to integrate on-premise, DataCore-managed and virtualized solid state and disk-based storage with the Microsoft Azure cloud, providing a powerful solution to tier to and access even larger off-premise scalable cloud storage. The appliances make it practical to automate off-site backups, offload inactive archive data from on-premises, and provide dedupe, compression, replication and mobility services to lower costs and accelerate disaster recovery. The combination of Microsoft Azure, StorSimple and DataCore helps meet the requirements of enterprises in need of a hybrid cloud strategy and infrastructure that seamlessly blends on-premise and private cloud resources with off-premise highly scalable public cloud resources.

DataCore SANsymphony-V software-defined storage and Virtual SAN software solutions are designed to maximize the performance, availability and utilization of IT assets. DataCore’s storage virtualization software provides a common platform that works across any storage hardware – pooling and protecting a mix of storage devices and brands (including flash, SAN or cloud), while centralizing management and automating many of the functions. DataCore’s sophisticated auto-tiering capability optimizes storage performance and automates the migration of data based on storage classifications and policies to maximize effective resource utilization across solid-state flash technologies, disks and cloud storage. The software allows organizations to dynamically select the most appropriate storage tier and paths to achieve their desired levels of performance and availability.

DataCore can access and auto-tier storage directly to the Microsoft StorSimple appliance, allowing it to  fully optimize on-site data storage resources while leveraging the cloud for elastic storage needs such as archival, remote-site backups, snapshot storage and disaster recovery workloads.

"A major challenge within the storage infrastructure today is the missing flexibility for growth," says Robert Gorbahn, Senior Solution Sales Professional, Microsoft StorSimple. "With Microsoft Azure StorSimple and SANsymphony-V customers build an extremely cost-efficient solution by integrating different storage classes. The most current StorSimple 8000 series extends every DataCore storage environment seamlessly with a Microsoft Azure cloud-integrated storage tier. The Logos "certified for Windows Server 2012 R2" and "DataCore ready" further prove the validation of these solutions."

“Our DataCore Ready Program identifies solutions that are verified to enhance DataCore software-defined storage infrastructures. The combination of DataCore and StorSimple appliances provides a seamless way to leverage the Microsoft Azure cloud as a tier of storage within the enterprise’s virtual storage pool,” said Paul Murphy, vice president of worldwide marketing, DataCore Software. “This provides a very flexible solution to meet the increasing and constantly changing needs of enterprises to cost-effectively offsite data protection, archive and offload corporate data to the cloud.” 

At Powering the Cloud 2014, DataCore previews latest release of its Virtual SAN and SANsymphony-V, and announces DataCore Ready Certification for Microsoft Azure StorSimple

Breakthrough in scalability and performance for ‘any hypervisor, any storage’ - DataCore Virtual SAN. 

DataCore this week will showcase its Virtual SAN solution and preview its latest product innovations at Powering the Cloud 2014 (Congress Center, Frankfurt a.M., Germany, 28-29 Oct., Stand G70).  DataCore will also highlight its growing network of strategic alliances with new DataCore Ready certified joint solutions from major partners like Fujitsu, Dell and Microsoft. The next release introduces new breakthroughs in scalability and performance, providing  more powerful and comprehensive enterprise-class functionality to DataCore’s  SANsymphony™-V10storage services platform and to its ‘any hypervisor, any storage’ DataCore™ Virtual SAN.

At Powering the Cloud 2014 DataCore will preview an important update to DataCore’s Software-defined Storage platformSANsymphony-V10 that will be made available in November 2014. The new SANsymphony-V10 and Virtual SAN update release features breakthroughs in the areas of scalability and performance and provides new innovations including a new capability designed to highly optimize random write processing, especially for transaction-oriented applications like databases and ERP systems.

DataCore’s new ‘any hypervisor, any storage’ Virtual SAN capabilities will be featured at the show. DataCore Virtual SANeliminates the need to deal with the traditional high costs and complexities of storage networks. Unlike alternative approaches, it is not bound to a single hypervisor platform or locked to specific storage hardware vendor’s system, instead it works cross-platform with all major hypervisors (VMware vSphere, KVM, Microsoft Hyper-V, etc.) and popular flash and storage device offerings. The new update will further enhance enterprise-class functionality and scaling of DataCore’s ‘any hypervisor, any storage’ Virtual SAN by doubling the number of nodes supported up to 64, boosting random IO performance and providing an extended growth path to integrate and unify physical SANs with SANsymphony-V making it one of the most powerful and scalable systems available on the market today.

Another important example of the growth and expansion of DataCore’s partner ecosystem is the strategic cooperation with Microsoft. At Powering the Cloud 2014, DataCore will deliver demos and presentations at their booth G70 on how to benefit from using DataCore Virtual SANs in Microsoft Hyper-V environments. The partners will use the event to announce a new certified solution that allows customers to use DataCore to manage and virtualize on-premise storage and do backups and tier workloads and archive storage to Microsoft Azure-based Cloud storage.

"Software is driving the future of storage and our software-defined “data anywhere” architecture enables organizations to manage, virtualize and leverage all resources from server flash and disk to the external SAN and to the Cloud from a single point spanning different departments, data centres and remote locations regardless of storage or server brands or hypervisor platforms”, says Christian Hagen, Senior Vice President EMEA, who will represent DataCore with European top level representatives at the branch leading event. “So we identified Powering the Cloud 2014 is an ideal platform to provide end users and partners with the first preview on our upcoming release of our flagship SANsymphony-V and Virtual SAN solutions and feature our newest certified and ready-to-go offerings that we have developed with strategic partners Fujitsu, Dell, Microsoft and many others.”

Free DataCore Virtual SAN licenses:

DataCore is offering free, non-production licenses of its Virtual SAN software intended primarily for technical specialists, virtualization consultants, certified storage experts, instructors and architects evaluating technologies to manage and optimize storage infrastructures. To request a copy and learn more, please see www.datacore.com/products/features/virtual-san.