Im OFTC IRC #ceph habe ich die Frage auch gestellt und folgende Konversation folgte darauf:
10∶19
ys Hi, i have a short question.
Can I run a CEPH OSD / OSN (ObjectStorageDevice / Node) locally on the client and force the pool (which contains the objects) or the CRUSH algorithm to have an RBD (RADOSBlockDevice) or file system completely on the local OSD retains and primarily uses it?
like a Roaming Profile or Offline Filesystem
user1 ys, no
ys ;(
user1 you can cache rbd's on the client using various tools tho.
but for offline access i think a filesystem type cache/sync would be better then block device. not aware of any block based cache's that support offline sync. perhaps you could jusy rig something based on NBD. but i think i would look at what you are actualy trying to do, and see if block based offline sync realy is a nessesarcy tool for that
user2 ceph is a cluster, not a local filesystem
so its not aimed nor useful for that laptop that is offline at times or something similar.
ys unfortunately there are no suitable or ready solutions for this
https://forum.ubuntuusers.de/topic/ceph-infrastruktur-herausforderung/
user1 ys, sorry that is greek to me
ys, what is it you are trying to acomplish
ys i would also use this functionality for virtual machines
I would like to share for VM block devices and for data filesystems.
VMs must be high-performing and data must be available offline. This is best done with a local copy.
user2 so, whats keeping you from having a local filesystem and sometimes syncing it to whatever remote place you want?
ys to much work :)
too much effort
user2 so you need a Kinderegg? Both a toy, a surprise and a piece of chocolate, all wrapped in a cheap casing and readily available close to the checkout in the food store. Sorry, ceph is not that.
ys there is no ready-made solution for linux as with windows for "roaming profile" and "offline files". I thought with this construct one could construct these functions
user2 on the places where my job tried roaming profiles, it would break if anyone put a CD ISO on the desktop, since it would sync that on every login/logout
ys :D
user2 if you intend to sync VM images like that, then this analogy would not hold
user3 I sitll dont get it
just use a rbd for the VM?
its all networkbased..
as user2s aid, I think you mistake ceph for soemthing else
its a HA storage system for enterprises
ys network performance should not affect the VM
user3 doesnt make sense
if its not stored locally it always has to affect the VM
ys and I figured I could make a multi-purpose solution out of it
user3 you're aware that you need at least 3 nodes for the smallest ceph cluster
and with default settings you only get 1/3 of storage
not sure what you trying todo but ceph aint for it
user2 having a single box should probably mean "zfs" and using "zfs send" of snapshots ever X time
ys not at least 2?
user2 ys: no
its a cluster
user3 default requires 3, 2 is possible but kinda defeats the purpose
also ceph is not for high performance
takes lot of money to make it fast
ys do you really think the idea is so absurd?
user2 ys: you are giving out conflicting or too vague demands
user3 I still dont understand the idea
you want some local but non-local storage for VM disk?
thats independent of network speed?
user2 that is probably why you dont see tons of answers going "you should use this, that or the third option of X,Y,Z"
user3 the question is just counterfactual
user2 neither here nor on that german forum
user3 either its local or its network-based
ys it is mainly about the synchronization and double storage of the data centrally on a NAS but also locally on a notebook or a VM host PC
fully automated
user2 "backups"
user3 theres network solutions. but sure they all depend on network speed etc, u lose performance
user2 something hidden that you dont want to tell us makes your idea "not backups", but when you try to tell us how simple it is, you describe backups
ys but not if I have a complete copy of the data locally on the client for reading.
The writing needs an acknoledge of n nodes. This can take a while
but could be configurable to n = 1 ?!
user3 sure but its not ceph
user2 n = 1 is "I have the files on local system"
user3 thats NFS etc
user2: not if its a network mount ;p
user2 nfs would not satisfy the double storage demand
user3 yea im ignoring that lol
user1 ys, local vm, with local disk, incremental backup to nas whenever laptop is network connected.
ys OK thanks. I just wanted feedback from knowledgeable and nice people about the feasibility.
That is a clear answer. thank you
user3 ys: its simply too many constrains, and yea ceph is not meant for it anyway
ys Pity. would have been a nice surprise Kinderegg
:)
user4 yo
ys @user3, @user1, @user2 may I post our conversation pseudonymized in the ubuntuusers.de forum?
to complete the topic in the forum
user1 sure, but i think a synopsis might suite the web forum format better.
ys, what you could do tho... was run a single node ceph cluster on your machine, then use the rbd mirror functionallity to async replicate the rbd to the datacenter ceph cluster. not sure what rbd-mirror would do if it lost conenction for long periodes of time. might need a kick to start again.
ys @user3, , @user2 ?
user1 and ofcourse would not performe as you expect. since ceph have the huge distributed overhead
that becomes extra problematic on a cluster of 1 :)
user2 lots of things become interesting in a cluster of 1
user1 yes :)
user2 I dont think the remote OSDs will be happy if the "main" one with the mon is gone for long times, the mons in turn will keep deltas on things as the remote OSDs are missing and so forth
user1 would have to be separate clusters.
with mons and everything
rbd mirror from dynamic ip to DC... would require an ip.
would function tho. but naturally be slow as molasses on a single node ceph cluster hybrid vm host laptop
Zusammengefasst wird davon abgeraten CEPH für diesen Einsatzzweck zu ge(miss)brauchen.
Daher werde ich diesen Ansatz nicht weiter verfolgen.