Disruptive Technologies for 2015: Software-defined Storage, Virtual SAN, Flash and Hybrid Cloud Predictions

From StorageNewsletter

George Teixeira

President and CEO, DataCore Software

Major storage hardware vendors saw flat growth or significant sales declines as a result of all the disruptive forces hitting the data storage market this year. Virtual SANs, hybrid clouds, flash for performance and software-defined storage have become the rage but enterprises are still struggling to implement these new technologies in a practical manner.  

Below are four related enterprise software predictions for 2015. 

#1: Software-defined Storage Will Go Mainstream in 2015

The value proposition for Software-defined storage is inevitable. It will continue to disrupt enterprise storage in 2015. It will further commoditize underlying storage devices and raises storage features and services from being locked to those devices. True software-defined storage platforms like DataCore will deliver comprehensive storage services that allow devices to achieve a new level of productivity by being able to work cross-platform and infrastructure-wide.  Bottom-line, the compelling economic benefits, better productivity and the need for greater agility to meet future requirements will drive software-defined storage to become a mainstream solution in 2015.

#2: Hybrid Clouds will become Practical in 2015

Enterprises are dealing with both on-premise storage and off-site cloud storage (hybrid cloud). This will become a bigger issue as customers become smarter about what cloud workloads are practical. On-premise storage is usually allocated for active data such as transaction-oriented business. The cloud will still be primarily used for cold data, back-up and disaster recovery due to internet speeds. New solutions are emerging such as DataCore and Microsoft StorSimple, which combine to allow data (from any storage) to be seamlessly migrated from on-premise to a cloud such as Microsoft Azure. This will fuel the larger trend, which is for enterprises to do a mix of on-premise and cloud. In addition, while doing disaster recovery from the cloud remains complex, new integration tools and more automated processes are on the way to make this a more practical solution.

#3: Disk and flash software stacks must span both worlds. Flash is excellent for specialized workloads that require high speed reads such as databases, but it is not a cost-effective solution for all workloads and still makes up a very small fraction of the installed storage base overall. On the other side of the spectrum are low-cost SATA disk drives that continue to advance and use new technologies like helium to support huge capacities, up to 10 TB per drive, but they are not highly performant and are slow.  Write-heavy transaction workloads also need to be addressed differently. (See New Breakthrough Random Write Acceleration and Impact on Disk Drives and Flash technologies). All flash is still not practical due to the costs involved, and the large installed base of disk storage that must be addressed. Comprehensive software stacks  will rise in importance in 2015, since users must be able to optimize the cost and performance trade-offs and migrate workloads to the right resources needed whether flash or disk. Software-defined storage done right can help unify the new world of flash with the existing and still-evolving world of disks. Both have a future.

#4: Servers as Hyper-converged Virtual SANs will Continue to Displace Traditional Storage Arrays

The latest generation of servers are powerful and will continue to support even larger amounts of storage. Software defined storage software such as DataCore’s Virtual SAN software will further drive servers to be transformed into powerful storage systems and, in effect, enterprise-class virtual SANs.

See the Dell 4Enterprise Blog post, “Dell PowerEdge Servers Make Great Software-Defined Storage Solutions,”  and the  “Software-Defined Virtual SAN Solutions Powered by FUJITSU PRIMERGY Servers” as examples of this trend. 

Happy Holidays!

DataCore's New Random Write Accelerator for Databases,SQL, ERP, OLTP and RAID-5 workloads

DataCore in its most recent release has added even more powerful and comprehensive enterprise-class functionality to its SANsymphony™-V10 storage services platform and its ‘any hypervisor, any storage’ DataCore™ Virtual SAN. The newest enhancements further boost I/O performance and double the scale of hyper-converged storage systems to 64 nodes.  

Accelerate Performance Up to 30X for Random Write Intensive Applications such as OLTP

The new software release includes a new Random Write Accelerator, which yields up to 30 times faster performance for random-write-heavy workloads that frequently update databases, ERP and OLTP systems. Even greater performance gains can be realized on RAID-5 protected datasets that spread data and reconstruction information to multiple locations across different disk drives.

To Learn more, see our webinar series and demo:

SSD-like Speeds from Spinning Disks; Faster Response and Reduced Write-wear for Flash
The remarkable write performance speed-ups are most pronounced when using low-cost spinning disks (e.g., SATA disk drives) where each write incurs significant rotational and seek delays to mechanically update blocks on spinning platters. Higher speed flash-based technologies (SSDs), especially prone to write-amplification and rapid wear, benefit from the Random Write Accelerator as well, delivering up to 3.6 times more IOPS.

Double the Scalability – Hyper-Converged Systems up to 64 Nodes and Over 100 Million IOPsThe new version of DataCore Virtual SAN supports larger scale configurations that can be managed centrally and organized to match business operational needs. With these latest enhancements, customers can now spread their workloads over more servers in a cluster and share resources across multiple clusters in order to improve their response and throughput. 

Doubling the possible number of nodes in a DataCore Virtual SAN is particularly valuable for latency-sensitive applications distributed over large-scale clusters, as well as virtual desktop (VDI) deployments for thousands of users.

Scaling out the configuration across more nodes enables better distribution of the workloads and segmentation of the physical storage capacity. DataCore’s zero-touch failover capabilities ensure workloads and applications are not disrupted in the event of a node or storage failure. The internode mirrors can also be stretched over metro-wide distances up to 60 miles (100 kilometers) to improve overall availability and business continuity. Scaling out to more nodes also enhances the overall resiliency of the infrastructure.

Press Release: 

DataCore Introduces Major Breakthroughs in Performance, Scalability and Quality of Service in its New Software-Defined Storage and Virtual SAN Release

SearchVirtualStorage: DataCore boosts performance, adds Storage Accelerator and QoS to SANsymphony-V10 and Virtual SAN

DataCore Software's SANsymphony-V10 now features a random write accelerator to improve performance of write-intensive workloads.

DataCore Software Corp. updated its SANsymphony-V10 storage virtualization software to boost performance and scalability, add quality of service capabilities and provide tighter integration with Microsoft Azure cloud services.

...The DataCore software runs on standard x86 servers and manages the capacity of their internal disk and/or solid-state drives (SSDs) as a shared storage pool.

One of the key features in the SANsymphony-V10 PSP1 update is the random write accelerator. DataCore said the new technology converts random writes into sequential writes to improve the performance of write-intensive workloads such as databases, ERP and online transaction processing systems, and workloads that use RAID 5 data protection.

Jon Toigo, CEO and managing principal at Toigo Partners International LLC, said DataCore could also position the random write accelerator to address the performance-degrading I/O blender phenomenon. The I/O blender effect occurs when multiple virtual machines try to write data at the same time.

"It's basically doing some log-file structuring and some write coalescence that turns a whole bunch of random writes into sequential writes so that you speed up the performance of your storage," said Toigo, who is a user of DataCore software.

Updates boost performance, management

Augie Gonzalez, director of product marketing at DataCore, said Iometer testing of 100% random write workloads with a 4K block size showed 33 times faster performance, with low-cost SATA hard disk drives (HDDs) running on Dell servers. According to Gonzalez, using identical test conditions, the random-write-accelerated SATA HDDs outperformed multi-level cell (MLC) SSDs that did not use the new "turbo charger" feature.

Gonzalez said, when using the random write accelerator, the SSDs also saw performance 3.6 times faster than without the write accelerator. The acceleration technology, coupled with the product's caching mechanism, also helps to reduce the write-induced wear-out factor of SSDs, he said.

SANsymphony-V10 PSP1 increases the maximum number of server nodes supported in hyper-converged configurations from 32 to 64. That allows customers to spread workloads across more servers in a cluster to boost throughput. The company said each server can support more than 1 million IOPS, and the extra scalability can be especially helpful with distributed latency-sensitive applications and large virtual desktop infrastructure (VDI) deployments.

"Let's say the production cluster is running out of space," Gonzalez said. "Before this customer can stand up another server and put more storage on it, they've got to get over this afternoon's processing window. Their test and dev cluster has extra space. Normally those two couldn't touch each other. In the case of DataCore, having a common storage pool, we can basically loan that production cluster some temporary capacity."

SANsymphony-V10 PSP1 also introduces quality of service (QoS) controls to enable customers to manage and regulate the system to ensure that low-priority workloads don't monopolize resources that more critical applications need. The new release features host group access control to let administrators assign different application hosts to different segments of the storage pool.

"The combination allows you to segregate local partitions of the data to where you need it but also to remap it when you need to," DataCore CEOGeorge Teixeira said.

Gonzalez noted that DataCore's QoS applies not only to capacity but also to Fibre Channel and iSCSI connections with the host. With PSP1, the company has added utilization tracking capabilities and chargeback reports to enable customers to manage their storage infrastructure as a private cloud.

Another new option in SANsymphony-V10 is the ability to have a standby node to enable full performance during software updates, equipment refreshes and maintenance tasks or to offload responsibilities from an overtaxed node. The product already supported automatic failover between nodes in the event of a storage or device failure, but there was a performance impact. DataCore customers will now have the option to shift workloads to a standby node without disrupting the applications or degrading performance and throughput.

Teixeira said the standby node will be especially helpful for customers such as banks or hospitals that use metro clusters or metro mirroring stretching across different locations. While this type of feature traditionally appeals to customers with no tolerance for failure, and who are willing to pay for an extra server to get an even higher level of availability, falling server prices could bring it to more midmarket IT organizations.

DataCore looks to public cloud for backup and DR

With the PSP1 update, DataCore also gives customers additional options to send data to public cloud storage for off-site backups, archives and disaster recovery. The company tightly integrated and certified its SANsymphony-V10 to work with on-premises Microsoft StorSimple hybrid cloud arrays, which can serve as a bridge to Microsoft's Azure public cloud storage.

"One of the advantages we bring to Microsoft is that their array only works with iSCSI storage," Teixeira said. "We work with both Fibre Channel and iSCSI, so we can virtualize all the storage on-premises, whether it is Fibre Channel or iSCSI based, and we can automatically tier and migrate it to the StorSimple gateway."

...Teixeira said he is especially focused on Windows Azure because the service caters to business use involving databases such as Microsoft SQL Server and Exchange Server.

"When we talk to our customers, what they're looking for is a rock-solid commercial capability out there, and Windows Azure seems to be that platform for most of them," Teixeira said. "So, getting it jointly certified was key for us. I think it's probably where we're going to see the most pickup from our customer base."

See Full Article at: http://searchvirtualstorage.techtarget.com/news/2240234858/DataCore-boosts-performance-adds-QoS-to-SANsymphony-V10 

ComputerWorld: House of Travel Australia finds value in DataCore Software-defined storage

Travel agent firm can scale its storage environment up or down when required

House of Travel Australia has used a software-defined storage services platform to reduce capital expenditure and improve the resilience of its storage environment.

The company has a network of 460 personal travel managers based around Australia.

House of Travel Australia's director of IT, Matthew Harris, said it implemented DataCore’s SANsymphony-V software in May 2014 to manage virtualized storage for its VMware ESX systems.

“The expenditure on this software solution saved significant capital expenditure while providing a stable operational environment,” he said.

“The biggest benefit is peace of mind for our future. We can scale [storage] up or down and scale to multiple sites. DataCore can handle any level of workload that I need to give to it."

House of Travel operates two business units, which are TravelManagers and HOOT Holidays. Both business units store their data on the DataCore solution.

Aside from giving Harris peace of mind, the system proved its worth when there was a hardware failure with one of its IBM servers earlier in 2014.

“The beauty of the DataCore system is that it keeps all of my data on two different servers. The [IBM] server went totally dead and there was no chance to do anything,” he said.

“DataCore indicated that the data was on another server and made it available to the VMware ESX system. As far as the VMware system was aware, it lost some processing and random access memory [RAM] power but all of its storage was still there. It automatically re-started our virtual machine on to the other hardware storage. The business suffered about one minute of sluggish performance but there was no downtime or data loss,” said Harris.

He is now looking at using DataCore’s Asynchronous offsite replication. This will provide an exact copy of House of Travel Australia’s data offsite so the firm can recover information quickly in an emergency. Harris plans to implement the software in Q1 of 2015.

More at: http://www.computerworld.com.au/article/560844/house-travel-australia-finds-value-software-defined-storage/

The Register: DataCore lifts baton, strikes up the 64th SANsymphony with latest update release for SANsymphony-V and Virtual SAN

http://www.theregister.co.uk/2014/11/27/datacore_drives_sansymphony_64_ways_twice/

Comment DataCore has updated its SANsymphony and Virtual SAN software products to reach 100 million IOPS and 64PB capacity. It's also getting into bed with Azure.

SANsymphony-V10 PSP1 builds a SAN from a cluster of servers, aggregating each server's storage into a central virtual pool. The company has built its product on virtual SAN technology which VMware has now blessed as a concept with its VSAN technology...

...DataCore says that - unlike VMware's ESXi-only VSAN - its Virtual SAN supports any hypervisor and any storage.

Its software enhancements mean the following things:

• Systems can scale out to 64 nodes, up from 32
• They can scale up to 64PB configurations
• Notionally they can deliver more than 100 million IOPS

Claiming a 64PB and 100 million IOPS capability is bold; achieving it will be impressive.

The SANsymphony V10 PSP1 product has quality of service (QoS) features providing policies which manage access to storage tiers divided between on-premise and public clouds, defining storage resource allocation and providing Service Level Agreement capability. This ensures high priority applications aren't slowed by less important ones whose consumption of resources is controlled by capping their data transfer rate and IOPS.

DataCore says storage resources can be logically segregated, monitored and regulated on a departmental basis. This SW release also adds utilisation tracking capabilities and chargeback reports.

Random Write Accelerator technology has been added, which - DataCore says - benefits ERP, OLTP and database applications with up to 30 times faster writes. This secret sauce is said to provide a greater than 30x random write improvement on RAID-5 protected datasets with data and metadata - reconstruction information - spread across multiple locations on different disk drives.

It is also said to be especially effective with slower SATA disk drives exhibiting longer rotational and seek delays when updating data blocks. And flash can benefit too, with SSDs having lower write amplification and less wear, as well as a claimed 3.6x increase in IOPS.

This sound to us like some kind of random write buffering transforming many random writes into fewer sequential ones.

Doubling the node count in Virtual SANs helps increase the number of applications they support or the number of users in the VDI use case. DataCore says resources can be shared across multiple clusters as well.

There is zero-touch failover to provide non-disruptive failover if either a storage or server component fails. Internet mirrors can span 60-mile (100 kilometre) distances.

Microsoft Azure integration

SANsymphony-V can use Microsoft's Azure cloud core via a Microsoft StorSimple array with a very, very long name - the DataCore-ready certified Azure StorSimple Hybrid Cloud Storage Array.

In a hybrid cloud scheme Azure provides backend geo-redundant cloud storage, with SANsmphony-V auto-tiering on-premise storage to StorSimple Arrays and hence Azure. The StorSimple arrays can be pooled like any other SanSymphony storage.

Working set backups (snapshot copies) can be sent to Azure through the StorSimple gateways. They can be retrieved from separate locations by other StorSimple arrays or by Azure-hosted applications.

If accessing hosts use Fibre Channel, SANsymphony-V converts this protocol to iSCSI.

Bits and bobs

There's more; SANSymphony V10 PSP1 gets performance-monitoring capabilities, wizards to automate processes such as software updates, and enhanced CDP capabilities for restoring from known good points-in-time.

There is now a standby node capability whereby workloads can be redistributed when an existing node is taken out of service or gets overburdened in a SANsymphony-V group.

The latest Fujitsu PRIMERGY BX and CX servers are certified as DataCore Ready Software-defined Storage servers. Fujitsu's ETERNUS DX arrays have gained DataCore Ready Certification.

SANsymphony V10 PSP1 and VirtualSAN are shipping now with the enhancements above. DataCore's 10,000 customer sites should be pleased with this raft of new features as their mature product gets significantly extended.