settingsLogin | Registersettings

[Openstack-operators] Operating Swift

0 votes

Hello all,

I've installed swift with 3 storage nodes and 1 proxy node for testing.
So far everything seems smooth but I have questions regarding to
operating swift.

The most important thing is how to add a new storage node or proxy node.
While installing, I copied the files created by ring builder to all of
the storage nodes. Should I add a the node to the ring and copy the
files to all the storage nodes? If so, should I restart all the swift
services so that they read the ring. Is there a reload functionality in
swift processes? Is there a guide for operating swift cluster? I would
be really happy if there is document explaining what to do when storage
node dies, how the data is replicated in swift, and how to add new
storage/proxy nodes to the cluser.

The second one is the hardware requirements for proxy and load balancer
services. Is there a rule of thumb, or best practice for choosing server
hardware for storage and proxy node. I've read that proxy node is
network and CPU intensive but as I don't have traffic, I don't know
which specification I should choose (for example 8 CPU, 32GB ram, etc).

Your experience is appreciated.

Regards,

--
Eren Türkay, System Administrator
https://skyatlas.com/ | +90 850 885 0357

Yildiz Teknik Universitesi Davutpasa Kampusu
Teknopark Bolgesi, D2 Blok No:107
Esenler, Istanbul Pk.34220


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

asked Mar 31, 2015 in openstack-operators by Eren_Türkay (1,220 points)   1 4 8

2 Responses

0 votes

Well this is a big topic, isn't it? :-)

You could fill a book with info like that. Like this book: http://shop.oreilly.com/product/0636920033288.do

So with the understanding that anything I say below will be limited by time and space (and is covered in more detail in the book above), here goes...

On Mar 30, 2015, at 11:25 PM, Eren Türkay erent@skyatlas.com wrote:

Hello all,

I've installed swift with 3 storage nodes and 1 proxy node for testing.
So far everything seems smooth but I have questions regarding to
operating swift.

Great!!

The most important thing is how to add a new storage node or proxy node.
While installing, I copied the files created by ring builder to all of
the storage nodes.

All that's necessary to distribute is the ring.gz files. You don't have to distribute the .builder files for the cluster operation (but don't lose the builder files!).

Should I add a the node to the ring and copy the
files to all the storage nodes?

Yes. But depending on how much capacity you're adding, you might want to do this gradually. See this post for more info https://swiftstack.com/blog/2012/04/09/swift-capacity-management/

If so, should I restart all the swift
services so that they read the ring.

The swift processes will automatically detect and reload new rings. For configs, you'll have to reload or restart.

Is there a reload functionality in
swift processes?

Yes. Use swift-init. You can restart, reload, start, and stop Swift processes with it.

Is there a guide for operating swift cluster? I would
be really happy if there is document explaining what to do when storage
node dies, how the data is replicated in swift, and how to add new
storage/proxy nodes to the cluser.

Definitely check the book above. Here's a few links with some more info:

http://swift.openstack.org/
https://swiftstack.com/openstack-swift/
https://www.youtube.com/watch?v=_sUvfGKhaMo (a talk I gave about some failure handing)
https://www.youtube.com/watch?v=vAEU0Ld-GIU (a talk I gave that's an intro to swift)

The second one is the hardware requirements for proxy and load balancer
services. Is there a rule of thumb, or best practice for choosing server
hardware for storage and proxy node. I've read that proxy node is
network and CPU intensive but as I don't have traffic, I don't know
which specification I should choose (for example 8 CPU, 32GB ram, etc).

This depends a lot on your use case. All I can say is test it and see what works best for you. Swift doesn't need custom hardware, and you can upgrade a live cluster with no client downtime (https://swiftstack.com/blog/2013/12/20/upgrade-openstack-swift-no-downtime/), so use what you have. Use what works.

--John


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Mar 31, 2015 by John_Dickinson (10,400 points)   2 4 7
0 votes

On 31-03-2015 10:02, John Dickinson wrote:
Well this is a big topic, isn't it? :-)

Hey John, yeah it's a big topic :)

You could fill a book with info like that. Like this book:
http://shop.oreilly.com/product/0636920033288.do

Thank you for your answer. Those links will be helpful for me. I will
order the book as well.

What I am planning now is to add at least 2 proxy nodes (with haproxy
setting, using vrrp in case of failure) and add 5 storage nodes. Proxy
nodes are connected to storage nodes via internal 10G interface, which I
believe will be enough for now.

I am trying to use the same set of hardware for storage nodes, and
different hardware for proxy nodes based on the need. I believe proxy
nodes will be CPU and network intensive as it will read data out of
storage and send back to the client. Storage nodes will have, as you
expect, lots of drives. The suggested RAM is 1G per 1TB. I will have
enough ram. Hopefully, it will work smoothly.

Thanks again for the links.

Regards,

--
Eren Türkay, System Administrator
https://skyatlas.com/ | +90 850 885 0357

Yildiz Teknik Universitesi Davutpasa Kampusu
Teknopark Bolgesi, D2 Blok No:107
Esenler, Istanbul Pk.34220


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Mar 31, 2015 by Eren_Türkay (1,220 points)   1 4 8
...