July 18, 2024


The Internet Generation

Data migration specialist Datadobi adds S3-to-S3 support

Data migration expert Datadobi is bolstering its DobiMigrate application with support for Dell EMC’s PowerStore midrange lineup and numerous object retailers that are compatible with Amazon’s S3 API.

Consumers can use Datadobi application to shift facts from non-Dell EMC storage arrays to the new PowerStore devices that introduced in Could. The extension of support to the most up-to-date Dell EMC storage will come as no shock. 4 former EMC engineers started Datadobi in 2010, and the Belgium-centered firm has labored carefully with EMC, and subsequently, Dell EMC, on facts migration from Centera and NAS devices.

The support for S3-to-S3 object facts migration with the DobiMigrate five.nine update, thanks on July thirteen, is extra of a novel twist for Datadobi. The DobiMigrate application could help consumers to change facts from Amazon’s Basic Storage Company (S3) to on-premises S3-centered object retailers, and vice versa. Datadobi programs to support AWS S3, Cloudian HyperStore, Dell EMC ECS, IBM Cloud Item Storage, NetApp StorageGrid and Scality Zenko CloudServer when DobiMigrate five.nine launches.

Datadobi CTO Carl D’Halluin said the firm would test and guarantee compatibility with other S3-centered object retailers centered on purchaser requests. Feasible selections include things like object retailers from Google, Backblaze and Wasabi to support consumers migrate facts between community clouds.

Datadobi DobiMigrate.
Datadobi’s DobiMigrate five.nine update provides the capability to shift facts between object retailers that support Amazon’s S3 API.

Data migration from AWS

But Michael Jack, Datadobi’s co-founder and VP of world-wide sales, expects the most well-known use circumstance could be AWS to on-premises object storage to support consumers steer clear of the egress service fees that Amazon and other cloud suppliers charge to access facts. A secondary resource of desire could be consumers refreshing their object storage for much better features or increased efficiency.

“There was a large flurry of media a calendar year and a 50 % in the past all over repatriation — that repatriation is likely to be substantial because Amazon’s too costly,” Jack said. But, somewhat than a “enormous tsunami of possibilities,” Jack expects to see “incremental business enterprise” from consumers that don’t want to be locked in to a distinct storage seller or storage.

A great deal of individuals get upset about the egress service fees that they failed to definitely recognize with community cloud storage.
Marc StaimerPresident, Dragon Slayer Consulting

“A great deal of individuals get upset about the egress service fees that they failed to definitely recognize with community cloud storage,” said Marc Staimer, president of Dragon Slayer Consulting. “They want to shift, but there’s no easy way to migrate from S3 to S3 natively, so they require a 3rd social gathering.”

Datadobi specializes in migrating massive portions of facts. The firm at first focused on transferring facts out of EMC’s Centera articles-addressable storage and then developed DobiMigrate to deal with NAS migration when EMC’s Isilon staff arrived at out. The DobiMigrate five.nine update will add connectors to S3 on major of the NAS migration framework, incorporating crucial lessons that Datadobi’s engineering staff acquired through its NAS get the job done about the incompatibilities and vagaries in different vendors’ devices.

S3 facts migration is not trivial

“We thought S3 would be basic, because it truly is a basic protocol, but the identical is true here,” D’Halluin said. “Copying around S3 facts is not trivial, and enterprises want some assures that the facts is copied around correctly.”

D’Halluin said Datadobi engineers experienced to “invent new strategies” to scan the S3 articles and help fast copying and parallel scanning. DobiMigrate runs on premises powering the facts center’s firewall, underneath the command of the facts heart administrator. The application can synchronize the objects and metadata from the resource buckets to the goal buckets.

“A single of the product’s values to the organization is the scale at which it can operate,” said Krista Macomber, a senior analyst at Evaluator Team. “This is crucial for enterprises on the lookout to migrate petabytes of facts. The scanning procedure is multithreaded, and first full, as well as incremental, scans come about exterior of the facts route. This allows to speed up the scanning and copying procedure and reduce down on the routine maintenance window.”

Macomber thinks extra Datadobi consumers could shift S3 facts to the cloud, for disaster recovery or archival functions or as aspect of a legacy software migration. But she said pulling facts back again on premises could also be crucial as business enterprise analytics will become extra well-known.

DobiMigrate supports only NAS-to-NAS or S3-to-S3 migrations. The application does not offer gateway providers to migrate file-centered NAS facts to S3-centered object retailers, or vice versa. Datadobi supports the migration of facts from NAS or Centera devices to S3 object retailers only on an unique job basis.

Datadobi roadmap

Datadobi roadmap objects include things like optimizing migrations from Amazon’s Glacier, chilly storage as well as support for superior object attributes these types of as locking and tagging. D’Halluin said there is no incentive yet to support migrations between S3-centered object retailers and Microsoft’s Azure because product support for Azure’s Blob object storage API is minimal.

D’Halluin said Datadobi programs to period in S3-to-S3 migration capabilities and support consumers analyze which facts to provide back again from AWS, preserve in the cloud or delete entirely. He expects consumers at first will require support, so Datadobi will offer the new S3-to-S3 features as a services before building it readily available through the do-it-yourself application.

DobiMigrate has a 50 TB least for S3-to-S3 migrations. Jack estimated the services listing rate would begin at about $forty five,000 for a 50 TB migration, depending on the customer’s alternative of storage seller, devices integrator or a benefit-included reseller to deliver it. Jack said the do-it-yourself application would listing at about $thirty,000 for an S3-to-S3 migration of 50 TB of facts.

Jack estimated that twenty five% of Datadobi’s consumers use the application on their very own, and 75% go through the company’s extra than 100 partners. Datadobi sells only through the channel, but Jack said the company’s sales business communicates directly with all consumers. In addition to DobiMigrate, Datadobi sells DobiReplicate for NAS-to-NAS file-centered replication and DobiSync for file-to-object synchronization.

Datadobi statements to have extra than 800 consumers, including services suppliers and enterprises in industries these types of as economic providers, health care, oil and gasoline, and media and amusement. They are inclined to have massive amounts of facts that can scale earlier 1 PB in advanced multivendor environments spanning on-premises and cloud web-sites, according to Jack.

Datadobi’s competition features absolutely free applications these types of as robocopy and rsync, as well as Data Dynamics and Komprise. Jack said Datadobi focuses on massive one-time, “fire-hose” types of migrations in comparison to the “trickle” strategy that Data Dynamics and Komprise take with their facts management/migration merchandise.

Staimer said StrongBox Data Options gives S3-to-S3 migration as a characteristic but finds that the portions of moved object-centered facts are inclined to be smaller in comparison to the multi-petabyte assignments the seller does with file-centered facts.