settingsLogin | Registersettings

[openstack-dev] [gnocchi][ceilometer] gnocchi-metricd error using redis as coordination

0 votes

Hi gnocchi devs,

I have an issue when using gnocchi 4.0, the storage backend is ceph, and
tooz coordination is redis. currently gnocchi api in apache wsgi mode,
only one controller node running gnocchi-metricd & gnocchi-statsd daemon.
the error log of gnocchi-metricd is as follow.

2017-08-05 18:14:18.643 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:18.654 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils [-] Unhandled
exception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils Traceback (most
recent call last):
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/cotyledon/
utils.py", line 84, in
exitonexception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils yield
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/cotyledon/service.py", line 139, in _run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils self.run()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 120, in run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils self.configure()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/tenacity/init.py", line 87, in
wrappedf
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils return r.call(f,
*args, **kw)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 177, in call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
fut.result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 396,
in result
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
self.__get_result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 159, in call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils result =
fn(*args, **kwargs)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 193, in _configure
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils self.GROUP_ID,
partitions=200)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/coordination.py", line 284, in
join_partitioned_group
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
partitioner.Partitioner(self, group_id)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/partitioner.py", line 45, in __init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
partitions=self.partitions)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 47, in __init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
self.add_nodes(set(nodes))
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 71, in add_nodes
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils UnicodeDecodeError:
'ascii' codec can't decode byte 0xde in position 4: ordinal not in
range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

Is this a config issue or bug ? any tips or help is much appreciated :-)

--
Tang Yaguang


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Aug 8, 2017 in openstack-dev by Yaguang_Tang (1,180 points)   1 3 4

3 Responses

0 votes

From the log info,It shows that your 'node' maybe is not the valid str.
You can show the node name via 'print node' and try to call
str(node).encode('utf-8')
, identify does it can goes well.

2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils UnicodeDecodeError:
'ascii' codec can't decode byte 0xde in position 4: ordinal not in
range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

On Sat, Aug 5, 2017 at 7:16 PM, Yaguang Tang heut2008@gmail.com wrote:

Hi gnocchi devs,

I have an issue when using gnocchi 4.0, the storage backend is ceph, and
tooz coordination is redis. currently gnocchi api in apache wsgi mode,
only one controller node running gnocchi-metricd & gnocchi-statsd daemon.
the error log of gnocchi-metricd is as follow.

2017-08-05 18:14:18.643 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:18.654 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils [-] Unhandled
exception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils Traceback (most
recent call last):
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/cotyledon/
utils.py", line 84, in
exitonexception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils yield
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/cotyledon/service.py", line 139, in
_run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils self.run()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 120, in run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils
self.configure()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/tenacity/init.py", line 87, in
wrappedf
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils return
r.call(f, *args, **kw)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 177, in call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
fut.result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 396,
in result
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
self.__get_result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 159, in call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils result =
fn(*args, **kwargs)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 193, in _configure
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils self.GROUP_ID,
partitions=200)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/coordination.py", line 284, in
join_partitioned_group
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
partitioner.Partitioner(self, group_id)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/partitioner.py", line 45, in
__init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
partitions=self.partitions)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 47, in __init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
self.add_nodes(set(nodes))
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 71, in add_nodes
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils UnicodeDecodeError:
'ascii' codec can't decode byte 0xde in position 4: ordinal not in
range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

Is this a config issue or bug ? any tips or help is much appreciated :-)

--
Tang Yaguang


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 7, 2017 by Along_Meng (260 points)  
0 votes

Thanks Along, finally I figure out that this is a bug and fixed by this
commit

commit e749b60f49a4a3b48cc5da67a797f717dd8cd01d
Author: Julien Danjou julien@danjou.info
Date: Tue Jun 20 16:36:14 2017 +0200

utils: use ASCII bytes as member id

Tooz actually wants ASCII bytes and not random bytes.

Fixes #130

diff --git a/gnocchi/utils.py b/gnocchi/utils.py
index f81d93e..7666711 100644
--- a/gnocchi/utils.py
+++ b/gnocchi/utils.py
@@ -90,7 +90,7 @@ def enablecoordination(coord):

def getcoordinatorandstart(url):
- my
id = uuid.uuid4().bytes
+ myid = str(uuid.uuid4()).encode()
coord = coordination.get
coordinator(url, myid)
_enable
coordination(coord)
return coord, my_id

On Mon, Aug 7, 2017 at 10:10 PM, Along Meng alongmeng@gmail.com wrote:

From the log info,It shows that your 'node' maybe is not the valid str.
You can show the node name via 'print node' and try to call str(node).encode('utf-8')
, identify does it can goes well.

2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils
UnicodeDecodeError: 'ascii' codec can't decode byte 0xde in position 4:
ordinal not in range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

On Sat, Aug 5, 2017 at 7:16 PM, Yaguang Tang heut2008@gmail.com wrote:

Hi gnocchi devs,

I have an issue when using gnocchi 4.0, the storage backend is ceph, and
tooz coordination is redis. currently gnocchi api in apache wsgi mode,
only one controller node running gnocchi-metricd & gnocchi-statsd daemon.
the error log of gnocchi-metricd is as follow.

2017-08-05 18:14:18.643 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:18.654 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils [-] Unhandled
exception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils Traceback (most
recent call last):
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/cotyledon/
utils.py", line 84, in
exitonexception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils yield
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/cotyledon/service.py", line 139, in
_run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils self.run()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 120, in run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils
self.configure()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/tenacity/init.py", line 87, in
wrappedf
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils return
r.call(f, *args, **kw)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 177, in
call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
fut.result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line
396, in result
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
self.__get_result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 159, in
call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils result =
fn(*args, **kwargs)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 193, in
_configure
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils self.GROUP_ID,
partitions=200)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/coordination.py", line 284, in
join_partitioned_group
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
partitioner.Partitioner(self, group_id)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/partitioner.py", line 45, in
__init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
partitions=self.partitions)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 47, in __init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
self.add_nodes(set(nodes))
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 71, in
add_nodes
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
UnicodeDecodeError: 'ascii' codec can't decode byte 0xde in position 4:
ordinal not in range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

Is this a config issue or bug ? any tips or help is much appreciated :-)

--
Tang Yaguang



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscrib
e
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Tang Yaguang


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Yaguang_Tang (1,180 points)   1 3 4
0 votes

On Tue, Aug 08 2017, Yaguang Tang wrote:

Yes, this fix is included in Gnocchi 4.0.1 already.

Thanks Along, finally I figure out that this is a bug and fixed by this
commit

commit e749b60f49a4a3b48cc5da67a797f717dd8cd01d
Author: Julien Danjou julien@danjou.info
Date: Tue Jun 20 16:36:14 2017 +0200

utils: use ASCII bytes as member id

Tooz actually wants ASCII bytes and not random bytes.

Fixes #130

diff --git a/gnocchi/utils.py b/gnocchi/utils.py
index f81d93e..7666711 100644
--- a/gnocchi/utils.py
+++ b/gnocchi/utils.py
@@ -90,7 +90,7 @@ def enablecoordination(coord):

def getcoordinatorandstart(url):
- my
id = uuid.uuid4().bytes
+ myid = str(uuid.uuid4()).encode()
coord = coordination.get
coordinator(url, myid)
_enable
coordination(coord)
return coord, my_id

On Mon, Aug 7, 2017 at 10:10 PM, Along Meng alongmeng@gmail.com wrote:

From the log info,It shows that your 'node' maybe is not the valid str.
You can show the node name via 'print node' and try to call str(node).encode('utf-8')
, identify does it can goes well.

2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils
UnicodeDecodeError: 'ascii' codec can't decode byte 0xde in position 4:
ordinal not in range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

On Sat, Aug 5, 2017 at 7:16 PM, Yaguang Tang heut2008@gmail.com wrote:

Hi gnocchi devs,

I have an issue when using gnocchi 4.0, the storage backend is ceph, and
tooz coordination is redis. currently gnocchi api in apache wsgi mode,
only one controller node running gnocchi-metricd & gnocchi-statsd daemon.
the error log of gnocchi-metricd is as follow.

2017-08-05 18:14:18.643 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:18.654 1329257 INFO gnocchi.storage.common.ceph [-] Ceph
storage backend use 'cradox' python library
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils [-] Unhandled
exception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils Traceback (most
recent call last):
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/cotyledon/
utils.py", line 84, in
exitonexception
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils yield
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/cotyledon/service.py", line 139, in
_run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils self.run()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 120, in run
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils
self.configure()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils File
"/usr/lib/python2.7/site-packages/tenacity/init.py", line 87, in
wrappedf
2017-08-05 18:14:19.100 1329257 ERROR cotyledon.
utils return
r.call(f, *args, **kw)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 177, in
call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
fut.result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line
396, in result
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
self.__get_result()
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 159, in
call
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils result =
fn(*args, **kwargs)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 193, in
_configure
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils self.GROUP_ID,
partitions=200)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/coordination.py", line 284, in
join_partitioned_group
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils return
partitioner.Partitioner(self, group_id)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/partitioner.py", line 45, in
__init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
partitions=self.partitions)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 47, in __init__
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
self.add_nodes(set(nodes))
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils File
"/usr/lib/python2.7/site-packages/tooz/hashring.py", line 71, in
add_nodes
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils key =
str(node).encode('utf-8')
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils
UnicodeDecodeError: 'ascii' codec can't decode byte 0xde in position 4:
ordinal not in range(128)
2017-08-05 18:14:19.100 1329257 ERROR cotyledon._utils

Is this a config issue or bug ? any tips or help is much appreciated :-)

--
Tang Yaguang



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscrib
e
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Julien Danjou
;; Free Software hacker
;; https://julien.danjou.info


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Aug 11, 2017 by Julien_Danjou (20,500 points)   2 4 6
...