Thursday, 2017-04-27

« Wednesday, 2017-04-26 Index Friday, 2017-04-28 »
*** d0ugal has quit IRC00:04
*** yassine has joined #openstack-oslo00:25
*** yassine is now known as Guest9668500:26
*** hongbin has quit IRC00:33
*** tovin07_ has joined #openstack-oslo00:43
*** sdague has quit IRC00:58
*** yamahata_ has quit IRC01:33
*** gcb has joined #openstack-oslo01:38
*** zhangguoqing has joined #openstack-oslo02:34
*** dave-mccowan has quit IRC02:36
*** dave-mccowan has joined #openstack-oslo02:42
*** dave-mcc_ has joined #openstack-oslo02:44
*** zhangguoqing has quit IRC02:45
*** dave-mccowan has quit IRC02:47
*** dave-mcc_ has quit IRC03:24
*** nicolasbock has quit IRC03:27
*** dimtruck is now known as zz_dimtruck03:37
*** syjulian has quit IRC04:06
*** yamahata_ has joined #openstack-oslo04:45
*** salv-orlando has joined #openstack-oslo04:51
*** syjulian has joined #openstack-oslo05:11
*** ihrachys has joined #openstack-oslo05:26
*** ctp has quit IRC05:34
*** ctp has joined #openstack-oslo05:34
*** e0ne has joined #openstack-oslo05:49
*** e0ne has quit IRC05:55
*** ihrachys has quit IRC05:58
*** amotoki has quit IRC06:19
*** amotoki has joined #openstack-oslo06:20
openstackgerritMerged openstack/oslo.messaging master: Try to fix TestSerializer.test_call_serializer failed randomly  https://review.openstack.org/45123906:41
*** zhangguoqing has joined #openstack-oslo06:42
*** zhangguoqing has joined #openstack-oslo06:42
*** zhangguoqing has quit IRC06:43
*** zhangguoqing has joined #openstack-oslo06:44
*** d0ugal has joined #openstack-oslo06:44
*** pcaruana has joined #openstack-oslo06:44
openstackgerritMehdi Abaakouk (sileht) proposed openstack/oslo.messaging master: tests: fix MultiStrOpt value  https://review.openstack.org/46041906:47
*** salv-orlando has quit IRC06:48
*** shardy has joined #openstack-oslo06:50
*** shardy has quit IRC06:50
*** shardy has joined #openstack-oslo06:51
*** jaosorior_away is now known as jaosorior06:51
*** salv-orlando has joined #openstack-oslo07:07
*** Nakato has quit IRC07:08
*** Nakato has joined #openstack-oslo07:08
*** tesseract has joined #openstack-oslo07:09
*** Nakato has quit IRC07:30
*** Nakato has joined #openstack-oslo07:31
*** e0ne has joined #openstack-oslo07:52
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-oslo08:00
*** lucas-afk is now known as lucasagomes08:17
*** amotoki has quit IRC08:22
*** sheeprine has quit IRC08:23
*** shardy has quit IRC08:27
*** sheeprine has joined #openstack-oslo08:27
*** amotoki has joined #openstack-oslo08:32
*** mhickey has joined #openstack-oslo08:37
*** jbadiapa has quit IRC08:39
*** tovin07_ has quit IRC08:42
*** jbadiapa has joined #openstack-oslo08:45
*** d0ugal has quit IRC08:45
*** e0ne has quit IRC08:48
*** e0ne has joined #openstack-oslo08:49
*** lpetrut has joined #openstack-oslo08:50
*** d0ugal has joined #openstack-oslo09:04
*** d0ugal has quit IRC09:04
*** d0ugal has joined #openstack-oslo09:04
*** zhangguoqing has quit IRC09:06
*** aarefiev_afk is now known as aarefiev09:16
*** syjulian has quit IRC09:17
*** amotoki has quit IRC09:21
*** sambetts|afk is now known as sambetts09:21
*** amotoki has joined #openstack-oslo09:22
*** amotoki has quit IRC09:23
*** e0ne has quit IRC09:30
*** e0ne has joined #openstack-oslo09:36
*** amotoki has joined #openstack-oslo09:41
*** cdent has joined #openstack-oslo09:43
*** zhangguoqing has joined #openstack-oslo09:48
*** salv-orl_ has joined #openstack-oslo09:49
*** salv-orlando has quit IRC09:52
*** lpetrut_ has joined #openstack-oslo09:58
*** lpetrut has quit IRC10:00
*** d0ugal has quit IRC10:01
*** nicolasbock has joined #openstack-oslo10:03
*** d0ugal has joined #openstack-oslo10:08
*** syjulian has joined #openstack-oslo10:13
*** shardy has joined #openstack-oslo10:17
*** d0ugal has quit IRC10:21
openstackgerritMehdi Abaakouk (sileht) proposed openstack/oslo.messaging master: Fix aliases deprecation  https://review.openstack.org/46048910:23
*** d0ugal has joined #openstack-oslo10:26
*** amotoki has quit IRC10:28
*** salv-orl_ has quit IRC10:38
*** d0ugal has quit IRC10:45
*** sdague has joined #openstack-oslo11:04
openstackgerritMerged openstack/oslo.messaging master: tests: fix MultiStrOpt value  https://review.openstack.org/46041911:04
*** d0ugal has joined #openstack-oslo11:05
*** dave-mccowan has joined #openstack-oslo11:09
*** ansmith has quit IRC11:18
*** lucasagomes is now known as lucas-hungry11:27
*** shardy is now known as shardy_lunch11:28
*** cdent has quit IRC11:30
*** amotoki has joined #openstack-oslo11:45
*** amotoki has quit IRC11:58
*** amotoki has joined #openstack-oslo12:01
*** shardy_lunch is now known as shardy12:04
*** takedakn has joined #openstack-oslo12:15
*** amotoki has quit IRC12:19
*** amotoki has joined #openstack-oslo12:25
*** amotoki has quit IRC12:34
openstackgerritMerged openstack/oslo.service master: Add min_interval to BackOffLoopingCall  https://review.openstack.org/45979012:36
*** zhangguoqing has quit IRC12:36
*** ansmith has joined #openstack-oslo12:45
*** kgiusti has joined #openstack-oslo12:48
*** ctp has quit IRC12:53
*** lucas-hungry is now known as lucasagomes12:53
*** ctp has joined #openstack-oslo12:54
*** zhangguoqing has joined #openstack-oslo12:54
*** amotoki has joined #openstack-oslo12:55
*** cdent has joined #openstack-oslo12:57
openstackgerritgordon chung proposed openstack/tooz master: sql: close connection for lock if not used  https://review.openstack.org/46022912:58
*** gordc has joined #openstack-oslo12:58
*** zhangguoqing has quit IRC13:01
*** gordc has quit IRC13:01
*** gordc has joined #openstack-oslo13:02
*** d0ugal has quit IRC13:04
*** zhangguoqing has joined #openstack-oslo13:06
*** Guest94155 has quit IRC13:10
*** salv-orlando has joined #openstack-oslo13:12
*** d0ugal has joined #openstack-oslo13:20
*** spzala has joined #openstack-oslo13:22
*** shardy has quit IRC13:36
openstackgerritgordon chung proposed openstack/tooz master: sql: close connection for lock if not used  https://review.openstack.org/46022913:37
openstackgerritMerged openstack-dev/pbr master: Make oslosphinx optional  https://review.openstack.org/44355513:44
*** shardy has joined #openstack-oslo13:45
*** takedakn has quit IRC13:51
sdaguedhellmann: so... curious thing here - http://logs.openstack.org/62/460062/4/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/e52ae45/logs/devstack@n-api.service.log.txt.gz#_Apr_27_12_26_46_78070514:07
sdaguethis is logging issues going to the journal.send14:07
openstackgerritStephen Finucane proposed openstack/oslo.policy master: Add Sphinx extension to pretty-print modules  https://review.openstack.org/46057214:12
*** hongbin has joined #openstack-oslo14:13
*** zz_dimtruck is now known as dimtruck14:13
*** salv-orlando has quit IRC14:23
openstackgerritgordon chung proposed openstack/tooz master: sql: close connection for lock if not used  https://review.openstack.org/46022914:27
*** d0ugal has quit IRC14:28
*** zhangguoqing has quit IRC14:31
*** zhangguoqing has joined #openstack-oslo14:32
*** d0ugal has joined #openstack-oslo14:32
*** zhangguoqing has quit IRC14:37
openstackgerritgordon chung proposed openstack/tooz master: simplify pgsql lock  https://review.openstack.org/46061114:37
*** salv-orlando has joined #openstack-oslo14:38
*** d0ugal has quit IRC14:39
*** lpetrut_ has quit IRC14:40
openstackgerritJulien Danjou proposed openstack/tooz master: doc: update heartbeat doc to use start_heart=True  https://review.openstack.org/46061514:41
*** d0ugal has joined #openstack-oslo14:48
mtreinishdhellmann: do you have any ideas why: https://review.openstack.org/#/c/459474/8/glance/image_cache/drivers/sqlite.py (the lockutils.set_defaults() bit) doesn't work?14:54
mtreinishI feel like I'm missing something obvious14:54
mtreinishthe if block is definitely running: http://logs.openstack.org/74/459474/8/check/gate-grenade-dsvm-neutron-ubuntu-xenial/2f43209/logs/new/screen-g-api.txt.gz#_2017-04-26_15_50_02_410 but it still fails after setting a fallback default14:54
*** d0ugal has quit IRC14:56
*** pcaruana has quit IRC15:05
*** ihrachys has joined #openstack-oslo15:07
*** boden has joined #openstack-oslo15:11
*** jaosorior is now known as jaosorior_away15:13
*** zigo has quit IRC15:17
*** d0ugal has joined #openstack-oslo15:21
openstackgerritMerged openstack/tooz master: simplify hashring node lookup  https://review.openstack.org/45249615:27
*** mhickey has quit IRC15:30
*** aarefiev is now known as aarefiev_afk15:35
*** SpamapS has quit IRC15:37
sdaguemtreinish: that's evaluating way too early right?15:37
*** SpamapS has joined #openstack-oslo15:37
mtreinishsdague: I tried doing it in the class init and it didn't work there either15:38
mtreinishsdague: I figured set the default early just in case15:38
mtreinishsdague: oh, or are you saying the CONF.image_cache_dir might not be set to anything yet, because it hasn't even read the config file at that stage15:43
sdagueyep15:43
sdagueit's really really early15:43
sdaguedefinitely one of the challenges of the global object is realizing when it's actually a thing15:44
sdagueI also think that for that reason, it's never actually going to work15:44
sdaguebecause the decorator might have evaled enough that you can't change it15:45
sdagueis there a reason that you wouldn't just force the lock dir, I thought that was a typical fatal issue already15:45
mtreinishit's the upgrade path, glance has never required the lock dir before15:46
mtreinishI was trying to make the graceful transition instead of forcing people to add it to their config file15:46
*** yamahata_ has quit IRC15:48
mtreinishsdague: if I have to break the new version old configs guarantee there's no avoiding it I guess15:49
*** salv-orl_ has joined #openstack-oslo15:50
sdaguemtreinish: is this only needed for wsgi?15:50
mtreinishsdague: yeah, running under wsgi there will be multiple processes normally and the sqlite init will race at startup15:51
sdaguealso... I completely never realized that glance had a secret sqlite db15:51
mtreinishheh, yeah I had no idea either until I hit the failure15:52
*** salv-orlando has quit IRC15:52
sdagueso... I would almost try to work around that without lockutils15:53
mtreinishsdague: just blame jaypipes: https://review.openstack.org/#/c/1064/ for not telling anyone :)15:54
mtreinishsdague: yeah that was my original idea, catch the conflict exception and just ignore it15:54
sdaguemtreinish: is that the typical driver?15:54
mtreinishsdague: but I was concerned that the setup might not be finished in the other process before the skipped one tries to do  something with the db15:55
sdaguemtreinish: yeh, I would honestly test for the existance of the file, if so don't create, but try to query it15:55
mtreinishsdague: I'm not sure, it is in the gate at least15:55
sdagueif the query fails, do a retry, and hopefully it's built in the other worker by then15:55
*** e0ne has quit IRC15:57
mtreinishsdague: I'll give it a try. At least it will get me around the setting config issue15:58
sdagueok, now I get to get to the bottom of weird things in systemd journal land15:58
mtreinishI still need to play with that patch on my cloud, I think it'll cleanup my logging a fair bit16:01
*** lucasagomes is now known as lucas-afk16:31
*** salv-orl_ has quit IRC16:36
*** browne has joined #openstack-oslo16:55
*** salv-orlando has joined #openstack-oslo17:01
*** harlowja has quit IRC17:08
*** cdent has quit IRC17:24
*** amotoki has quit IRC17:32
*** ansmith has quit IRC17:34
*** tongli has joined #openstack-oslo17:41
*** ansmith has joined #openstack-oslo17:49
*** harlowja has joined #openstack-oslo17:51
*** erlon has joined #openstack-oslo17:57
*** shardy has quit IRC18:00
*** tongli has quit IRC18:06
*** spzala has quit IRC18:15
*** salv-orlando has quit IRC18:27
*** e0ne has joined #openstack-oslo18:29
*** ansmith has quit IRC18:31
*** john51_ has quit IRC18:34
*** john51 has joined #openstack-oslo18:34
*** fnordahl_ has joined #openstack-oslo18:38
*** dhellmann has quit IRC18:38
*** fnordahl has quit IRC18:38
*** dhellmann has joined #openstack-oslo18:39
*** ansmith has joined #openstack-oslo18:47
*** openstackgerrit has quit IRC18:48
*** cdent has joined #openstack-oslo19:01
*** salv-orlando has joined #openstack-oslo19:30
*** mhickey has joined #openstack-oslo19:32
*** spzala has joined #openstack-oslo19:43
dhellmannmtreinish, sdague : IIRC, you can call set_defaults() at any point and if no value is present in the file the new default will be used when the option is accessed.19:58
dhellmannit appears that the config option is accessed each time a lock is acquired?19:58
dhellmannso you'd just need to move that set_defaults() call to somewhere in the runtime that happens before the lock is acquired19:58
dhellmannbut after cfg.CONF is fully initialized so that image_cache_dir is defined19:59
dhellmannif image_cache_dir is not defined in the sqlite.py file, you probably want to use import_opt to make sure the definition is available19:59
mtreinishdhellmann: hmm, my lastest attempt moved it to an inner function and calls set default before that's defined: https://review.openstack.org/#/c/459474/20:02
mtreinishI assumed that would work because the option is definitely used before the set_defaults call20:03
*** e0ne has quit IRC20:03
dhellmannwhat's the failure mode? does it get an empty path?20:04
mtreinishdhellmann: http://logs.openstack.org/74/459474/9/check/gate-grenade-dsvm-neutron-ubuntu-xenial/6118863/logs/new/screen-g-api.txt.gz#_2017-04-27_19_44_55_05920:04
dhellmanninteresting20:06
* dhellmann thinks20:07
mtreinishdhellmann: the thing that sticks out to me there is that it's complaining about it not being in [DEFAULT], but in lockutils.py it uses [oslo_concurrency] as the group20:09
dhellmannoh, that's a bug in lockutils not passing the right group name20:10
dhellmannwhen the exception is raised20:10
*** e0ne has joined #openstack-oslo20:10
dhellmannimage_cache_dir doesn't have a default. are you sure it's set in the config file?20:10
dhellmannI'd be interested in seeing the value in that log message you added, to see if it's set properly20:11
mtreinishdhellmann: yeah it is set, self.base_dir there is an alias for it20:12
mtreinishdhellmann: also: http://logs.openstack.org/74/459474/9/check/gate-grenade-dsvm-neutron-ubuntu-xenial/6118863/logs/etc/glance/glance-api.conf.txt.gz20:12
mtreinishdhellmann: but I can add it to the log message on the next rev20:12
dhellmannoh, this is a wsgi app20:14
*** sambetts is now known as sambetts|afk20:14
dhellmannmtreinish : do you know where in the setup chain for this the config file is actually being read?20:15
*** openstackgerrit has joined #openstack-oslo20:16
openstackgerritMerged openstack/tooz master: sql: close connection for lock if not used  https://review.openstack.org/46022920:16
mtreinishdhellmann: are you looking for where the parse gets called? If so it's here: https://github.com/openstack/glance/blob/master/glance/cmd/api.py#L7320:17
mtreinishthat's just a wrapper (I don't know why) https://github.com/openstack/glance/blob/master/glance/common/config.py#L675-L68020:18
dhellmannok, yeah20:18
mtreinishwait is it not setting the config files anywhere?20:19
dhellmannoslo.config builds some default paths20:19
*** cdent has quit IRC20:19
mtreinishoh, ok20:19
dhellmannhttp://logs.openstack.org/74/459474/9/check/gate-grenade-dsvm-neutron-ubuntu-xenial/6118863/logs/new/screen-g-api.txt.gz#_2017-04-27_19_44_55_057 seems to imply that it got an explicit instruction to use the sql driver, and it's not using that default path in image_cache/__init__.py20:20
dhellmannoh, although the default for the option is the sqlite driver20:20
dhellmannso we don't actually know20:20
dhellmannthat log message just means it's successfully loading the plugin, I guess20:21
dhellmannoh, hmm20:22
dhellmannI think I see why this is happening20:22
dhellmannmtreinish : ^^20:22
dhellmannget_lock_path() is going to access the option and cache the results20:22
dhellmannthen the default is changed on the Opt object20:23
dhellmannthen the attribute is used again20:23
mtreinishoh20:23
dhellmannbut at that point the cached value is used, not the default in the Opt20:23
mtreinishis there a way to manually flush the cache?20:23
dhellmannI think you just want to call lockutils.set_defaults() without testing first20:23
dhellmannit only changes the default, not the actual value20:23
dhellmannif there is a value in the config file, it will be used to override that default anyway20:23
mtreinishok, that makes sense20:23
dhellmannis this the only part of glance that uses lockutils?20:24
mtreinishI still want to warn if it's not set (because I think we'll want to make it mandatory in the future)20:24
mtreinishit's the only part using an external lock20:24
dhellmannok20:24
dhellmanncall set_defaults() first, then ask what the path is and test it against what you set the default to be20:24
dhellmannoh, no, that won't work if they explicitly set it to the same path20:24
mtreinishdhellmann: unless the user explictly sets it to that20:24
mtreinishheh, yep20:25
*** kgiusti has quit IRC20:25
dhellmannthere's no way to reset only one config value in the cache20:25
dhellmannyou have to throw away everything, including any registered options20:26
mtreinishhmm, yeah then that won't work20:26
dhellmannright20:26
mtreinishI can workaround the explictly setting image_cache_dir == lock_path in documentation20:26
dhellmannyou could phrase the warning in such a way that it won't be alarming20:26
mtreinishyeah20:27
openstackgerritMerged openstack/tooz master: doc: update heartbeat doc to use start_heart=True  https://review.openstack.org/46061520:27
dhellmannYou could also not warn at all, and just use that path as the default forever.20:27
mtreinishdhellmann: oh, that's probably fine too20:28
mtreinishI can't think of any potential conflict between the 2 paths20:28
mtreinishand if we're using the driver image_cache_dir should always be set20:28
dhellmannyou could log at info, too. "Setting the default lock_path to ..."20:28
dhellmannyeah, if that's already a constraint that's being enforced you'd be ok20:29
mtreinishdhellmann: ok pushed a new rev that just sets the default to that without a get or warning20:31
* mtreinish crosses his fingers20:31
* dhellmann crosses his fingers, too20:31
*** e0ne has quit IRC20:34
*** ansmith has quit IRC20:35
*** dimtruck is now known as zz_dimtruck20:37
*** salv-orlando has quit IRC20:39
*** salv-orlando has joined #openstack-oslo20:39
*** tongli has joined #openstack-oslo20:59
*** zz_dimtruck is now known as dimtruck21:20
*** tongli has quit IRC21:22
*** zigo has joined #openstack-oslo21:32
openstackgerritMike Perez proposed openstack/oslosphinx master: Build Common Framework for Feature Classification Matrix  https://review.openstack.org/33002721:34
mtreinishdhellmann: \o/ that worked, thanks for the help21:36
*** yamahata_ has joined #openstack-oslo21:36
*** gordc has quit IRC21:44
*** salv-orl_ has joined #openstack-oslo21:50
*** Nakato has quit IRC21:52
*** salv-orlando has quit IRC21:52
*** Nakato has joined #openstack-oslo21:53
*** salv-orl_ has quit IRC22:01
*** cburgess has quit IRC22:12
*** erlon has quit IRC22:13
*** peterstac has quit IRC22:14
*** harlowja has quit IRC22:14
*** rha has quit IRC22:15
*** cburgess has joined #openstack-oslo22:16
*** mhickey has quit IRC22:17
*** nikhil has quit IRC22:17
*** rushiagr has quit IRC22:17
*** xgerman has quit IRC22:17
*** purplerbot has quit IRC22:19
*** rha has joined #openstack-oslo22:20
*** ansmith has joined #openstack-oslo22:25
*** nikhil has joined #openstack-oslo22:26
*** rushiagr has joined #openstack-oslo22:32
*** salv-orlando has joined #openstack-oslo22:37
*** peterstac has joined #openstack-oslo22:37
*** boden has quit IRC22:47
*** dimtruck is now known as zz_dimtruck23:00
*** zz_dimtruck is now known as dimtruck23:04
*** salv-orlando has quit IRC23:08
*** sdague has quit IRC23:08
*** ihrachys has quit IRC23:09
*** tesseract has quit IRC23:20
*** dimtruck is now known as zz_dimtruck23:22
*** harlowja has joined #openstack-oslo23:23
*** hongbin has quit IRC23:29
« Wednesday, 2017-04-26 Index Friday, 2017-04-28 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!