*** luksky has quit IRC | 00:00 | |
*** macz_ has joined #openstack-ansible | 00:01 | |
*** macz_ has quit IRC | 00:03 | |
*** macz_ has joined #openstack-ansible | 00:04 | |
*** macz_ has quit IRC | 00:23 | |
*** spotz has quit IRC | 01:36 | |
*** rh-jelabarre has quit IRC | 02:24 | |
*** evrardjp has quit IRC | 02:33 | |
*** evrardjp has joined #openstack-ansible | 02:33 | |
*** mathlin has quit IRC | 03:57 | |
*** mathlin has joined #openstack-ansible | 03:58 | |
*** rohit02 has joined #openstack-ansible | 04:33 | |
*** rohit02 has quit IRC | 04:38 | |
*** rohit02 has joined #openstack-ansible | 04:40 | |
*** rohit02 has quit IRC | 04:41 | |
*** rohit02 has joined #openstack-ansible | 04:42 | |
*** rohit02 has quit IRC | 04:42 | |
*** rohit02 has joined #openstack-ansible | 04:43 | |
*** rohit02 has quit IRC | 04:48 | |
*** rohit02 has joined #openstack-ansible | 04:48 | |
*** rohit02 has quit IRC | 04:55 | |
*** shyamb has joined #openstack-ansible | 05:08 | |
*** shyam89 has joined #openstack-ansible | 05:08 | |
*** macz_ has joined #openstack-ansible | 05:23 | |
*** macz_ has quit IRC | 05:24 | |
*** yasemind has joined #openstack-ansible | 05:46 | |
openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Add trove tempest testing https://review.opendev.org/c/openstack/openstack-ansible/+/784379 | 05:59 |
---|---|---|
*** crazzy has joined #openstack-ansible | 06:11 | |
openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone stable/ussuri: Use absolute path for uwsgi_params include https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/785393 | 06:25 |
openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_horizon stable/ussuri: Fix race condition in compression of static files https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/775592 | 06:26 |
*** shyam89 has quit IRC | 06:52 | |
*** shyamb has quit IRC | 06:52 | |
*** shyamb has joined #openstack-ansible | 06:59 | |
*** shyam89 has joined #openstack-ansible | 06:59 | |
*** jbadiapa has joined #openstack-ansible | 06:59 | |
*** luksky has joined #openstack-ansible | 07:03 | |
*** rpittau|afk is now known as rpittau | 07:33 | |
*** andrewbonney has joined #openstack-ansible | 07:38 | |
*** ChanServ has quit IRC | 07:58 | |
*** andrewbonney has quit IRC | 07:58 | |
*** NewJorg has quit IRC | 07:58 | |
*** MrClayPole has quit IRC | 07:58 | |
*** LowKey has quit IRC | 07:59 | |
*** pcaruana has quit IRC | 07:59 | |
*** janno has quit IRC | 07:59 | |
*** strattao has quit IRC | 07:59 | |
*** gary_perkins_ has quit IRC | 07:59 | |
*** mugsie has quit IRC | 07:59 | |
*** cloudnull has quit IRC | 07:59 | |
*** dasp has quit IRC | 07:59 | |
*** trident has quit IRC | 07:59 | |
*** johanssone has quit IRC | 07:59 | |
*** Jeffrey4l has quit IRC | 07:59 | |
*** fresta_ has quit IRC | 07:59 | |
*** owalsh has quit IRC | 07:59 | |
*** priteau has quit IRC | 07:59 | |
*** odyssey4me has quit IRC | 07:59 | |
*** fridtjof[m] has quit IRC | 07:59 | |
*** gmann has quit IRC | 07:59 | |
*** CeeMac has quit IRC | 07:59 | |
*** noonedeadpunk has quit IRC | 07:59 | |
*** nicolasbock has quit IRC | 07:59 | |
*** jrosser has quit IRC | 07:59 | |
*** mwhahaha has quit IRC | 07:59 | |
*** jroll has quit IRC | 07:59 | |
*** persia has quit IRC | 07:59 | |
*** Carcer has quit IRC | 07:59 | |
*** rndmh3ro has quit IRC | 07:59 | |
*** PrinzElvis has quit IRC | 07:59 | |
*** fyx has quit IRC | 07:59 | |
*** sri_ has quit IRC | 07:59 | |
*** snadge has quit IRC | 07:59 | |
*** crazzy has quit IRC | 07:59 | |
*** logan- has quit IRC | 07:59 | |
*** bradm has quit IRC | 07:59 | |
*** zbr has quit IRC | 07:59 | |
*** sc has quit IRC | 07:59 | |
*** zigo has quit IRC | 07:59 | |
*** kukacz has quit IRC | 07:59 | |
*** rpittau has quit IRC | 07:59 | |
*** NobodyCam has quit IRC | 07:59 | |
*** cyberpear has quit IRC | 07:59 | |
*** mgagne has quit IRC | 07:59 | |
*** jungleboyj has quit IRC | 07:59 | |
*** Daemoen has quit IRC | 07:59 | |
*** guilhermesp has quit IRC | 07:59 | |
*** tbarron has quit IRC | 07:59 | |
*** fungi has quit IRC | 07:59 | |
*** johnsom has quit IRC | 07:59 | |
*** mnaser has quit IRC | 07:59 | |
*** melwitt has quit IRC | 07:59 | |
*** openstackgerrit has quit IRC | 07:59 | |
*** dirk has quit IRC | 07:59 | |
*** ebbex has quit IRC | 07:59 | |
*** mathlin has quit IRC | 07:59 | |
*** lemko has quit IRC | 07:59 | |
*** frickler has quit IRC | 07:59 | |
*** ianychoi has quit IRC | 07:59 | |
*** antonym has quit IRC | 07:59 | |
*** tobberydberg has quit IRC | 07:59 | |
*** partlycloudy has quit IRC | 07:59 | |
*** bverschueren_ has quit IRC | 07:59 | |
*** jmccrory has quit IRC | 07:59 | |
*** yasemind has quit IRC | 07:59 | |
*** cp- has quit IRC | 07:59 | |
*** kleini_ has quit IRC | 07:59 | |
*** djhankb has quit IRC | 07:59 | |
*** arxcruz has quit IRC | 07:59 | |
*** chandankumar has quit IRC | 07:59 | |
*** jhesketh has quit IRC | 07:59 | |
*** mcarden has quit IRC | 07:59 | |
*** corvus has quit IRC | 07:59 | |
*** Anticimex has quit IRC | 07:59 | |
*** luksky has quit IRC | 07:59 | |
*** Underknowledge has quit IRC | 07:59 | |
*** prometheanfire has quit IRC | 07:59 | |
*** brad[] has quit IRC | 07:59 | |
*** manti has quit IRC | 07:59 | |
*** tinwood has quit IRC | 07:59 | |
*** gouthamr has quit IRC | 07:59 | |
*** admin0 has quit IRC | 07:59 | |
*** jbadiapa has quit IRC | 07:59 | |
*** akahat has quit IRC | 07:59 | |
*** d34dh0r53 has quit IRC | 07:59 | |
*** hindret has quit IRC | 07:59 | |
*** jamesdenton has quit IRC | 07:59 | |
*** dpawlik has quit IRC | 07:59 | |
*** mubix has quit IRC | 07:59 | |
*** shyam89 has quit IRC | 07:59 | |
*** shyamb has quit IRC | 07:59 | |
*** evrardjp has quit IRC | 07:59 | |
*** dave-mccowan has quit IRC | 07:59 | |
*** jonher has quit IRC | 07:59 | |
*** mgariepy has quit IRC | 07:59 | |
*** rodolof has quit IRC | 07:59 | |
*** waxfire has quit IRC | 07:59 | |
*** poopcat has quit IRC | 08:03 | |
*** owalsh has joined #openstack-ansible | 08:04 | |
*** fresta_ has joined #openstack-ansible | 08:04 | |
*** Jeffrey4l has joined #openstack-ansible | 08:04 | |
*** johanssone has joined #openstack-ansible | 08:04 | |
*** trident has joined #openstack-ansible | 08:04 | |
*** ChanServ has joined #openstack-ansible | 08:04 | |
*** verne.freenode.net sets mode: +o ChanServ | 08:04 | |
*** pcaruana has joined #openstack-ansible | 08:04 | |
*** janno has joined #openstack-ansible | 08:04 | |
*** strattao has joined #openstack-ansible | 08:04 | |
*** gary_perkins_ has joined #openstack-ansible | 08:04 | |
*** mugsie has joined #openstack-ansible | 08:04 | |
*** cloudnull has joined #openstack-ansible | 08:04 | |
*** dasp has joined #openstack-ansible | 08:04 | |
*** evrardjp has joined #openstack-ansible | 08:04 | |
*** dave-mccowan has joined #openstack-ansible | 08:04 | |
*** jonher has joined #openstack-ansible | 08:04 | |
*** mgariepy has joined #openstack-ansible | 08:04 | |
*** rodolof has joined #openstack-ansible | 08:04 | |
*** pto has joined #openstack-ansible | 08:05 | |
*** jbadiapa has joined #openstack-ansible | 08:05 | |
*** akahat has joined #openstack-ansible | 08:05 | |
*** d34dh0r53 has joined #openstack-ansible | 08:05 | |
*** hindret has joined #openstack-ansible | 08:05 | |
*** jamesdenton has joined #openstack-ansible | 08:05 | |
*** dpawlik has joined #openstack-ansible | 08:05 | |
*** poopcat has joined #openstack-ansible | 08:05 | |
*** cp- has joined #openstack-ansible | 08:05 | |
*** kleini_ has joined #openstack-ansible | 08:05 | |
*** djhankb has joined #openstack-ansible | 08:05 | |
*** arxcruz has joined #openstack-ansible | 08:05 | |
*** chandankumar has joined #openstack-ansible | 08:05 | |
*** mcarden has joined #openstack-ansible | 08:05 | |
*** jhesketh has joined #openstack-ansible | 08:05 | |
*** corvus has joined #openstack-ansible | 08:05 | |
*** Anticimex has joined #openstack-ansible | 08:05 | |
*** andrewbonney has joined #openstack-ansible | 08:05 | |
*** NewJorg has joined #openstack-ansible | 08:05 | |
*** MrClayPole has joined #openstack-ansible | 08:05 | |
*** LowKey has joined #openstack-ansible | 08:05 | |
*** admin0 has joined #openstack-ansible | 08:05 | |
*** gouthamr has joined #openstack-ansible | 08:05 | |
*** tinwood has joined #openstack-ansible | 08:05 | |
*** Carcer has joined #openstack-ansible | 08:06 | |
*** persia has joined #openstack-ansible | 08:06 | |
*** jroll has joined #openstack-ansible | 08:06 | |
*** mwhahaha has joined #openstack-ansible | 08:06 | |
*** jrosser has joined #openstack-ansible | 08:06 | |
*** nicolasbock has joined #openstack-ansible | 08:06 | |
*** noonedeadpunk has joined #openstack-ansible | 08:06 | |
*** CeeMac has joined #openstack-ansible | 08:06 | |
*** gmann has joined #openstack-ansible | 08:06 | |
*** fyx has joined #openstack-ansible | 08:06 | |
*** PrinzElvis has joined #openstack-ansible | 08:06 | |
*** sri_ has joined #openstack-ansible | 08:06 | |
*** snadge has joined #openstack-ansible | 08:06 | |
*** jmccrory has joined #openstack-ansible | 08:06 | |
*** bverschueren_ has joined #openstack-ansible | 08:06 | |
*** partlycloudy has joined #openstack-ansible | 08:06 | |
*** tobberydberg has joined #openstack-ansible | 08:06 | |
*** antonym has joined #openstack-ansible | 08:06 | |
*** mathlin has joined #openstack-ansible | 08:06 | |
*** lemko has joined #openstack-ansible | 08:06 | |
*** frickler has joined #openstack-ansible | 08:06 | |
*** ianychoi has joined #openstack-ansible | 08:06 | |
*** crazzy has joined #openstack-ansible | 08:06 | |
*** bradm has joined #openstack-ansible | 08:06 | |
*** logan- has joined #openstack-ansible | 08:06 | |
*** zbr has joined #openstack-ansible | 08:06 | |
*** sc has joined #openstack-ansible | 08:06 | |
*** zigo has joined #openstack-ansible | 08:06 | |
*** mgagne has joined #openstack-ansible | 08:06 | |
*** kukacz has joined #openstack-ansible | 08:06 | |
*** rpittau has joined #openstack-ansible | 08:06 | |
*** tbarron has joined #openstack-ansible | 08:06 | |
*** NobodyCam has joined #openstack-ansible | 08:06 | |
*** cyberpear has joined #openstack-ansible | 08:06 | |
*** jungleboyj has joined #openstack-ansible | 08:06 | |
*** Daemoen has joined #openstack-ansible | 08:06 | |
*** guilhermesp has joined #openstack-ansible | 08:06 | |
*** fungi has joined #openstack-ansible | 08:06 | |
*** johnsom has joined #openstack-ansible | 08:06 | |
*** mnaser has joined #openstack-ansible | 08:06 | |
*** melwitt has joined #openstack-ansible | 08:06 | |
*** openstackgerrit has joined #openstack-ansible | 08:06 | |
*** dirk has joined #openstack-ansible | 08:06 | |
*** ebbex has joined #openstack-ansible | 08:06 | |
*** priteau has joined #openstack-ansible | 08:06 | |
*** odyssey4me has joined #openstack-ansible | 08:06 | |
*** cyberpear has quit IRC | 08:07 | |
*** jrosser has quit IRC | 08:07 | |
*** mwhahaha has quit IRC | 08:07 | |
*** yasemind has joined #openstack-ansible | 08:07 | |
*** mubix has joined #openstack-ansible | 08:07 | |
*** gouthamr has quit IRC | 08:08 | |
*** sri_ has quit IRC | 08:08 | |
*** mwhahaha has joined #openstack-ansible | 08:09 | |
*** gouthamr has joined #openstack-ansible | 08:09 | |
*** sri_ has joined #openstack-ansible | 08:09 | |
*** tosky has joined #openstack-ansible | 08:09 | |
*** jrosser has joined #openstack-ansible | 08:10 | |
*** cyberpear has joined #openstack-ansible | 08:13 | |
*** gouthamr has quit IRC | 08:20 | |
*** gouthamr has joined #openstack-ansible | 08:20 | |
*** mwhahaha has quit IRC | 08:20 | |
*** mwhahaha has joined #openstack-ansible | 08:20 | |
*** sri_ has quit IRC | 08:20 | |
*** sri_ has joined #openstack-ansible | 08:20 | |
*** jrosser has quit IRC | 08:20 | |
*** jrosser has joined #openstack-ansible | 08:20 | |
*** rndmh3ro has joined #openstack-ansible | 08:40 | |
*** luksky has joined #openstack-ansible | 08:49 | |
*** Underknowledge has joined #openstack-ansible | 08:49 | |
*** prometheanfire has joined #openstack-ansible | 08:49 | |
*** brad[] has joined #openstack-ansible | 08:49 | |
*** shyamb has joined #openstack-ansible | 09:06 | |
*** shyam89 has joined #openstack-ansible | 09:06 | |
*** fridtjof[m] has joined #openstack-ansible | 09:19 | |
*** manti has joined #openstack-ansible | 09:19 | |
*** Adri2000 has joined #openstack-ansible | 09:41 | |
*** mpjetta has joined #openstack-ansible | 09:45 | |
*** mpjetta has quit IRC | 09:47 | |
*** mpjetta has joined #openstack-ansible | 09:48 | |
*** mpjetta has quit IRC | 09:53 | |
openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: [goal] Deprecate the JSON formatted policy file https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/781537 | 10:03 |
pto | I have some issues with one of three cinder api containers (ceph storage backend), the two other containers sees the third as offline. Anyone here have seen something similar? | 11:02 |
noonedeadpunk | Is ceph itself healthy? | 12:11 |
*** shyamb has quit IRC | 12:11 | |
*** shyam89 has quit IRC | 12:11 | |
*** rh-jelabarre has joined #openstack-ansible | 12:15 | |
openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts stable/victoria: Add hostname resolution to deploy host https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/785597 | 12:30 |
*** mgariepy has quit IRC | 12:33 | |
*** mgariepy has joined #openstack-ansible | 12:47 | |
*** spatel_ has joined #openstack-ansible | 13:02 | |
*** spatel_ is now known as spatel | 13:02 | |
pto | noonedeadpunk: Yep. Everything works fine. Ceph health is OK, and cinder works perfectly. However, the ZennOss monitoring repports one of the containers goes down every 2 min ish, and from the logs on the other cinder API containers, they also loose connectiity | 13:05 |
noonedeadpunk | So it's whole container goes down, or systemd service or networking on the container? | 13:06 |
noonedeadpunk | or it's openstack volume service list shows it as down? | 13:07 |
pto | noonedeadpunk: All logs in journald looks fine on the affected container and it apperas to be up. But the two other cinder api containers repport the container as down: Apr 09 07:30:27 os-infra-d-01-cinder-api-container-e04515b8 cinder-scheduler[82]: 2021-04-09 07:30:27.405 82 WARNING cinder.scheduler.host_manager [req-29fa2061-d624-4ac9-8e31-4ac71aea43db 2930c19516714ae59827cb36839b6da8 3137e0dcce3946c0abda6cd1b53b1312 - default d | 13:08 |
pto | efault] volume service is down. (host: os-infra-e-01.srv.aau.dk@RBD) | 13:08 |
pto | https://pastebin.com/wELX6et5 | 13:08 |
noonedeadpunk | I think then openstack volume service list should be flapping as well... | 13:10 |
noonedeadpunk | but have no idea why this maight happen, sorry :( | 13:10 |
*** rpittau is now known as rpittau|bbl | 13:29 | |
*** spatel has quit IRC | 14:18 | |
*** macz_ has joined #openstack-ansible | 14:46 | |
*** d34dh0r53 has quit IRC | 15:04 | |
*** yasemind has quit IRC | 15:08 | |
*** d34dh0r53 has joined #openstack-ansible | 15:12 | |
*** evrardjp has quit IRC | 16:00 | |
*** evrardjp has joined #openstack-ansible | 16:05 | |
*** spatel_ has joined #openstack-ansible | 16:46 | |
*** spatel_ is now known as spatel | 16:46 | |
*** dwilde has joined #openstack-ansible | 16:55 | |
*** dwilde has quit IRC | 17:25 | |
*** spatel has quit IRC | 17:29 | |
*** andrewbonney has quit IRC | 17:34 | |
*** dwilde has joined #openstack-ansible | 17:39 | |
*** dwilde has quit IRC | 17:43 | |
*** dwilde has joined #openstack-ansible | 17:52 | |
*** spatel_ has joined #openstack-ansible | 17:56 | |
*** spatel_ is now known as spatel | 17:56 | |
*** spotz has joined #openstack-ansible | 18:40 | |
*** grabes has joined #openstack-ansible | 19:01 | |
*** lvdombrkr has joined #openstack-ansible | 19:14 | |
lvdombrkr | hello guys. someone have implemented 2key auth. in openstack? | 19:17 |
openstackgerrit | Merged openstack/openstack-ansible-os_octavia stable/victoria: Omit amp_ssh_access_allowed and remove amp_image_id options. https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/784614 | 19:21 |
*** lvdombrkr has quit IRC | 19:40 | |
*** spatel has quit IRC | 19:51 | |
*** spatel_ has joined #openstack-ansible | 19:54 | |
*** spatel_ is now known as spatel | 19:54 | |
spatel | Doing minor upgrade and hit this during bootstrap http://paste.openstack.org/show/804358/ | 19:58 |
jrosser | spatel: see this https://github.com/openstack/openstack-ansible/blob/master/scripts/get-ansible-role-requirements.yml#L123-L149 | 20:06 |
jrosser | there are parallel and non parallel git clones there | 20:06 |
jrosser | can you comment out the parallel one so that it uses the old method? | 20:06 |
spatel | why do we have two method ? | 20:07 |
jrosser | i'd like to know if the issue is with the parallel method | 20:07 |
jrosser | or if there is something to do with the repo structures which would fail with either | 20:08 |
spatel | ok let me comment | 20:08 |
jrosser | we have two methods because the parallel one is maybe 20x faster | 20:08 |
spatel | oh! got it | 20:08 |
spatel | also this is one time process generally so no need for speed :) | 20:09 |
jrosser | not for AIO | 20:09 |
spatel | i see | 20:10 |
jrosser | 10mins sat waiting each time you boot one | 20:10 |
jrosser | this is the error cmdline: git fetch --depth=20 --force\n stderr: 'fatal: error in object: unshallow 710a7095a93e23d13b8a611c01324e078180c2a3 | 20:10 |
spatel | jrosser all good after removing parallel block | 20:15 |
jrosser | huh | 20:15 |
spatel | its using Clone git repos (with git) | 20:15 |
spatel | or may be we can put smart checks around try one git fetch and if works then use otherwise silently move to rescue block | 20:17 |
jrosser | i'm not convinced of the value of shallow clones here | 20:18 |
jrosser | which i think is the underlying issue | 20:18 |
spatel | ok | 20:18 |
spatel | why are we not seeing that issue with CI jobs? | 20:18 |
jrosser | it puts much much more load on the git server to compute the shallow clone server side | 20:18 |
jrosser | becasue in the CI jobs the repos are already on the CI nodes | 20:19 |
jrosser | and they're just symlinked in | 20:19 |
spatel | hmm | 20:19 |
spatel | i think if its too much then lets remove parallel block until we have very good reason to keep it | 20:20 |
jrosser | the good reason is to save a bunch of time for building AIO | 20:20 |
jrosser | it might be better to switch the default round and make an extra flag to use parallel | 20:20 |
spatel | how much time? 10min or 20min? | 20:20 |
jrosser | it's really about 10mins just sat waiting for the repos to clone | 20:21 |
spatel | for AIO its ok to wait instead of seeing error first day :) | 20:21 |
spatel | I gotta go.. i will catch you on Monday, Thank you so much for helping me out. You have a wonderful weekend. | 20:23 |
jrosser | you too | 20:23 |
*** spatel has quit IRC | 20:29 | |
*** rpittau|bbl is now known as rpittau | 20:39 | |
*** luksky has quit IRC | 22:04 | |
*** rpittau is now known as rpittau|afk | 22:04 | |
*** macz_ has quit IRC | 22:50 | |
*** tosky has quit IRC | 23:29 | |
*** jamesdenton has quit IRC | 23:37 | |
*** jamesdenton has joined #openstack-ansible | 23:44 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!