If S3 you can use distcp from HDFS to S3@Ceph.
For example:
hadoop distcp -Dmapred.job.queue.name=queue_name Dfs.s3a.access.key=
-Dfs.s3a.secret.key= -Dfs.s3a.endpoint=
-Dfs.s3a.connection.ssl.enabled=false_or_true /hdfs_path/ s3a://path/
Regards
--
Jarek
pt., 6 lis 2020 o 12:29 Szabo, Istvan (Agoda) <Istvan.Szabo(a)agoda.com>
napisał(a):
Objectstore
On 2020. Nov 6., at 15:41, Jaroslaw Owsiewski <
jaroslaw.owsiewski(a)allegro.pl> wrote:
Email received from outside the company. If in doubt don't click links nor
open attachments!
________________________________
Hi,
What protocol do you want to make this data available on the Ceph?
--
Jarek
pt., 6 lis 2020 o 04:00 Szabo, Istvan (Agoda) <Istvan.Szabo(a)agoda.com
<mailto:Istvan.Szabo@agoda.com>> napisał(a):
Hi,
Is there anybody tried to migrate data from Hadoop to Ceph?
If yes what is the right way?
Thank you
________________________________
This message is confidential and is for the sole use of the intended
recipient(s). It may also be privileged or otherwise protected by copyright
or other legal rules. If you have received it by mistake please let us know
by reply email and delete it from your system. It is prohibited to copy
this message or disclose its content to anyone. Any confidentiality or
privilege is not waived or lost by any mistaken delivery or unauthorized
disclosure of the message. All messages sent to and from Agoda may be
monitored to ensure compliance with company policies, to protect the
company's interests and to remove potential malware. Electronic messages
may be intercepted, amended, lost or deleted, or contain viruses.
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io<mailtolto:
ceph-users-leave(a)ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io