*** watersj has quit IRC | 00:49 | |
*** cdearborn has quit IRC | 01:13 | |
*** k_mouza has joined #openstack-kolla | 01:31 | |
*** weshay|ruck is now known as weshay | 01:31 | |
*** k_mouza has quit IRC | 01:35 | |
*** noxoid has quit IRC | 01:37 | |
*** zhanglong has joined #openstack-kolla | 01:40 | |
bline | yoctozepto: I meant i couldn't find an entry for the failure but I did finally find a log entry that corresponds to each failure but it doesn't make much sense. | 03:37 |
---|---|---|
bline | ==> /var/log/kolla/barbican/barbican-api.log <== | 03:38 |
bline | -- unavailable modifier requested: 0 -- | 03:38 |
*** igordc has quit IRC | 03:42 | |
*** skramaja has joined #openstack-kolla | 03:50 | |
*** ykarel|away is now known as ykarel | 04:33 | |
*** evrardjp has quit IRC | 04:36 | |
*** evrardjp has joined #openstack-kolla | 04:37 | |
*** cah_link has joined #openstack-kolla | 04:45 | |
*** cah_link has quit IRC | 04:52 | |
*** threestrands has joined #openstack-kolla | 05:06 | |
*** cah_link has joined #openstack-kolla | 06:19 | |
*** mnasiadka has quit IRC | 06:21 | |
*** mnasiadka has joined #openstack-kolla | 06:22 | |
mnasiadka | morning | 06:22 |
*** vishalmanchanda has joined #openstack-kolla | 06:30 | |
*** abdysn has joined #openstack-kolla | 06:31 | |
yoctozepto | bline: whoa, it really has only one line in there? :O | 06:33 |
yoctozepto | mnasiadka: morning :D | 06:33 |
*** jaicaa has quit IRC | 06:33 | |
mnasiadka | morning yoctozepto :) | 06:33 |
*** jaicaa has joined #openstack-kolla | 06:35 | |
yoctozepto | mnasiadka: monasca-grafana again? it started yesterday; I'm pretty sure it got broken again | 06:36 |
mnasiadka | oh shit | 06:36 |
mnasiadka | I'll bug Doug when he'll be online | 06:36 |
yoctozepto | mnasiadka: shit indeed | 06:37 |
mnasiadka | I'm not keen on even having a quick look into that monster | 06:37 |
yoctozepto | it's npm failing, my world really ;p | 06:37 |
yoctozepto | bline: I just noticed this is the same message as in https://bugs.launchpad.net/kolla-ansible/+bug/1870086 btw, which release is that? | 06:39 |
openstack | Launchpad bug 1870086 in kolla-ansible "key-manager-service(Barbican)deployment should use wsgi with Apache" [Undecided,Incomplete] | 06:39 |
*** zhanglong has quit IRC | 06:39 | |
*** zhanglong has joined #openstack-kolla | 06:40 | |
yoctozepto | mnasiadka: https://review.opendev.org/713628 - fellow citizens getting upset ;D | 06:41 |
patchbot | patch 713628 - kolla-ansible (stable/rocky) - rabbitmq: update cluster version on reconfigure - 9 patch sets | 06:41 |
*** Abdallahyas has joined #openstack-kolla | 06:41 | |
mnasiadka | yoctozepto: well, rocky is EM, they need to patient :) | 06:42 |
*** abdysn has quit IRC | 06:44 | |
*** gary_perkins has quit IRC | 06:55 | |
*** Tony31 has joined #openstack-kolla | 06:57 | |
*** gary_perkins has joined #openstack-kolla | 06:58 | |
*** jbadiapa has joined #openstack-kolla | 07:01 | |
whoami-rajat | Hi #openstack-kolla , can i have a review on https://review.opendev.org/#/c/714391/ ? Thanks! | 07:05 |
patchbot | patch 714391 - kolla - Add nfs dependencies to glance container - 3 patch sets | 07:05 |
osmanlicilegi | morning | 07:06 |
*** rpittau|afk is now known as rpittau | 07:17 | |
*** diga has joined #openstack-kolla | 07:19 | |
*** bengates has joined #openstack-kolla | 07:21 | |
*** bengates has quit IRC | 07:27 | |
*** bengates has joined #openstack-kolla | 07:27 | |
yoctozepto | mnasiadka: ^^ -> whoami-rajat | 07:30 |
mnasiadka | yoctozepto: just commented | 07:31 |
openstackgerrit | Alfredo Moralejo proposed openstack/kolla master: DNM Test new deps based on final CBS builds https://review.opendev.org/710240 | 07:35 |
mnasiadka | yoctozepto: so openstack client is also broken in rocky... https://zuul.opendev.org/t/openstack/build/b068cd7cd18e4d1a90cff4462e7104dc/log/primary/logs/ansible/deploy#24950 | 07:44 |
yoctozepto | mnasiadka: yeah, I'm updating whiteboard live time | 07:45 |
yoctozepto | bad luck ;d | 07:45 |
openstackgerrit | Rajat Dhasmana proposed openstack/kolla master: Add nfs dependencies to glance container https://review.opendev.org/714391 | 07:52 |
whoami-rajat | yoctozepto, mnasiadka done. thanks for the reviews. | 07:54 |
mnasiadka | yoctozepto: how's the OVN change review? :) | 07:54 |
*** threestrands has quit IRC | 08:01 | |
*** ykarel is now known as ykarel|lunch | 08:08 | |
yoctozepto | mnasiadka: just like zun and tls :-) | 08:11 |
mnasiadka | yoctozepto: there's a zun and tls change? :D | 08:11 |
*** e0ne has joined #openstack-kolla | 08:15 | |
yoctozepto | mnasiadka: indeed, indeed | 08:29 |
*** iniazi has quit IRC | 08:32 | |
hrw | morning | 08:41 |
*** hjensas has quit IRC | 08:50 | |
mgoddard | morning | 08:59 |
*** pbing19 has joined #openstack-kolla | 09:00 | |
*** hjensas has joined #openstack-kolla | 09:01 | |
mgoddard | hrw, mnasiadka, osmanlicilegi, yoctozepto: anything I should be aware of that happened this week? | 09:09 |
hrw | mgoddard: I fooled several people in AArch64 circles with https://afd.biz.pl/ website of Polish startup making interesting AArch64 mainboard. Other than that probably nothing. | 09:10 |
hrw | cause me moving to Vancouver was posted only on Facebook and only for friends | 09:11 |
mnasiadka | mgoddard: yesterday I fixed CentOS 8 mirrors on Infra side (if you noticed some missing packages), monasca-grafana image is FUBAR in all branches | 09:11 |
mnasiadka | hrw: Vancouver? | 09:11 |
mgoddard | hrw: is this an April fool or real? | 09:11 |
hrw | fool of course | 09:12 |
mgoddard | :) | 09:12 |
mnasiadka | lol | 09:12 |
mnasiadka | :) | 09:12 |
hrw | mnasiadka: yes. I got an offer to move there. from Amazon | 09:12 |
mnasiadka | hrw: nice | 09:12 |
mgoddard | hrw: oh, so it's not a joke | 09:13 |
mgoddard | when are you moving? | 09:13 |
*** abdysn has joined #openstack-kolla | 09:15 | |
hrw | meh. | 09:16 |
*** Abdallahyas has quit IRC | 09:18 | |
*** pbing19 has quit IRC | 09:20 | |
yoctozepto | "Yet another clone of Raspberry/Pi. Nothing to be really proud of. But was well paid." legit desc | 09:20 |
*** pbing19 has joined #openstack-kolla | 09:32 | |
openstackgerrit | Merged openstack/kolla-ansible master: Add glance_ca_certificates_file when using self sign cert in glance https://review.opendev.org/715126 | 09:38 |
openstackgerrit | Merged openstack/kolla-ansible master: Add clients ca_file in heat https://review.opendev.org/715127 | 09:38 |
openstackgerrit | Merged openstack/kolla-ansible master: kolla-toolbox container name variable https://review.opendev.org/715200 | 09:38 |
openstackgerrit | Merged openstack/kolla-ansible master: Fix HAProxy prechecks during scale-out with limit https://review.opendev.org/714950 | 09:38 |
*** ykarel|lunch is now known as ykarel | 09:42 | |
hrw | https://marcin.juszkiewicz.com.pl/2020/04/02/april-fools-day/ | 09:55 |
hrw | yoctozepto: all pages removed | 09:55 |
yoctozepto | hrw: sorry to see it go, great bait that was! | 10:00 |
*** cah_link has quit IRC | 10:04 | |
*** zhanglong has quit IRC | 10:08 | |
*** k_mouza has joined #openstack-kolla | 10:12 | |
*** rpittau is now known as rpittau|bbl | 10:14 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Add clients ca_file in heat https://review.opendev.org/716913 | 10:15 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Add glance_ca_certificates_file when using self sign cert in glance https://review.opendev.org/716914 | 10:15 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Fix HAProxy prechecks during scale-out with limit https://review.opendev.org/716915 | 10:20 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: Fix HAProxy prechecks during scale-out with limit https://review.opendev.org/716917 | 10:21 |
openstackgerrit | Will Szumski proposed openstack/kayobe master: Switch to stackhpc fork of resmo.ntp https://review.opendev.org/715893 | 10:26 |
*** negronjl has quit IRC | 10:32 | |
*** negronjl has joined #openstack-kolla | 10:35 | |
yoctozepto | monasca-grafana builds locally so must be issue with infra npm mirror | 10:48 |
*** gfidente has joined #openstack-kolla | 11:09 | |
*** Abdallahyas has joined #openstack-kolla | 11:21 | |
*** abdysn has quit IRC | 11:24 | |
*** sorin-mihai__ has joined #openstack-kolla | 11:28 | |
*** sorin-mihai_ has quit IRC | 11:30 | |
*** cah_link has joined #openstack-kolla | 11:34 | |
mnasiadka | yoctozepto: another mirror issue, yay | 11:37 |
mnasiadka | yoctozepto: test it with a change to disable usage of infra npm mirror? :) | 11:39 |
*** sorin-mihai_ has joined #openstack-kolla | 11:45 | |
Wellie | CRITICAL freezer-scheduler [-] Unhandled error: ValueError: Unable to determine the name of the local timezone -- you must explicitly specify the name of the local timezone. Please refrain from using timezones like EST to prevent problems with daylight saving time. Instead, use a locale based timezone name (such as Europe/Helsinki). | 11:46 |
Wellie | kolla/ubuntu-source-freezer-scheduler | 11:46 |
*** sorin-mihai__ has quit IRC | 11:47 | |
*** ykarel is now known as ykarel|afk | 12:03 | |
*** rpittau|bbl is now known as rpittau | 12:12 | |
*** ykarel|afk is now known as ykarel | 12:13 | |
mgoddard | Wellie: related bug: https://bugs.launchpad.net/kolla-ansible/+bug/1821592 | 12:29 |
openstack | Launchpad bug 1821592 in kolla-ansible ussuri "Kolla Watcher Service - Unable to determine the name of the local timezone" [Medium,In progress] - Assigned to Dincer Celik (osmanlicilegi) | 12:29 |
yoctozepto | mnasiadka, mgoddard: https://review.opendev.org/716969 | 12:29 |
patchbot | patch 716969 - opendev/system-config - Do not canonicalize URLs to registry.npmjs.org - 1 patch set | 12:29 |
mgoddard | Wellie: proposed fix: https://review.opendev.org/#/c/704258 | 12:30 |
patchbot | patch 704258 - kolla-ansible - Introduce /etc/timezone to containers - 8 patch sets | 12:30 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Introduce /etc/timezone to containers https://review.opendev.org/704258 | 12:32 |
*** ykarel is now known as ykarel|afk | 12:33 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Introduce /etc/timezone to containers https://review.opendev.org/704258 | 12:36 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Avoid unconditional fact gathering https://review.opendev.org/716340 | 12:39 |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Switch to stackhpc fork of resmo.ntp https://review.opendev.org/715893 | 12:45 |
yoctozepto | mgoddard, mnasiadka: any of you on the kolla-toolbox breakage? | 12:55 |
*** ykarel|afk is now known as ykarel | 12:55 | |
yoctozepto | guessing not... | 12:56 |
mgoddard | yoctozepto: I was going to ask the same :) | 12:58 |
*** bersace has joined #openstack-kolla | 12:58 | |
mgoddard | yoctozepto: I could take a look. | 12:59 |
yoctozepto | yeah, I'm already | 13:00 |
yoctozepto | but could do paralellelly | 13:00 |
yoctozepto | nothing suspicious so far | 13:00 |
mgoddard | a two pronged attack | 13:00 |
yoctozepto | here goes nothing ;D | 13:02 |
*** sorin-mihai__ has joined #openstack-kolla | 13:03 | |
*** sorin-mihai_ has quit IRC | 13:05 | |
yoctozepto | got it | 13:06 |
yoctozepto | https://pypi.org/project/openstacksdk/#history | 13:06 |
yoctozepto | 0.45.0 | 13:06 |
mgoddard | yoctozepto: kolla-toolbox stein doesn't use UC | 13:06 |
yoctozepto | mgoddard: yeah, that's the problem ;/ | 13:07 |
*** sorin-mihai__ has quit IRC | 13:09 | |
mgoddard | yoctozepto: missing https://review.opendev.org/#/c/642416/ | 13:10 |
patchbot | patch 642416 - kolla - Bump up pkg versions in kolla-toolbox (MERGED) - 31 patch sets | 13:10 |
mgoddard | or part of it at least | 13:10 |
yoctozepto | mgoddard: yeah, full bump is a no-no | 13:10 |
mgoddard | could be an interesting one. I made a comment which was unanswered: | 13:11 |
mgoddard | > I think we will get some complaints from people behind air gaps if | 13:11 |
mgoddard | > we do not at least make the URL configurable. | 13:11 |
mgoddard | > | 13:11 |
mgoddard | > Ideally in source builds we'd use the requirements repo, but this | 13:11 |
mgoddard | > image is a bit of a fudge at the moment anyway. | 13:11 |
mgoddard | :) | 13:11 |
mgoddard | would be mean to introduce that on a stable branch | 13:11 |
yoctozepto | yeah, it's a fudge | 13:11 |
mgoddard | I'll have a crack at it | 13:12 |
yoctozepto | yeah, try this route | 13:12 |
yoctozepto | I'm trying why openstacksdk got intsalled on py2 | 13:12 |
yoctozepto | probably missing metadata... | 13:12 |
yoctozepto | two-pronged attack it is | 13:13 |
yoctozepto | :D | 13:13 |
mgoddard | yeah, I think something related went into ironic recently | 13:13 |
mgoddard | lots of packages seem to be missing this piece of metadata though | 13:13 |
mgoddard | yoctozepto: https://review.opendev.org/#/c/715454/2/setup.cfg | 13:15 |
patchbot | patch 715454 - ironic - Cleanup Python 2.7 support - 2 patch sets | 13:15 |
yoctozepto | mgoddard: yeah, it did not go into openstacksdk which miserably affected us as well | 13:15 |
mgoddard | yoctozepto: I guess the damage is done now - there is an incompatible release | 13:16 |
yoctozepto | mgoddard: yeah, hard to unwind | 13:18 |
mgoddard | yoctozepto: maybe simpler solution than UC is to pin to <0.45? | 13:18 |
kplant | any known issues with docker-ce 19.03.8 for k-a? | 13:18 |
mgoddard | and safer | 13:19 |
yoctozepto | mgoddard: just 0.27.0 | 13:19 |
yoctozepto | like in uc | 13:19 |
yoctozepto | most safe | 13:19 |
mgoddard | yeah | 13:19 |
mgoddard | although if other clients are unpinned could we hit incompatibilities with such different versions? | 13:20 |
Wellie | is openvswitch synchronized with each other? | 13:20 |
yoctozepto | mgoddard: I see most stuff is pinned | 13:20 |
*** jimcrowleyibm has joined #openstack-kolla | 13:21 | |
yoctozepto | ah no, you might be right | 13:21 |
yoctozepto | if they release py3-only improperly then we are also doomed | 13:21 |
*** jimcrowleyibm is now known as jimcrowleyibm[aw | 13:21 | |
yoctozepto | we can take all necessary pins from those u-c | 13:22 |
yoctozepto | python-novaclient | 13:22 |
yoctozepto | python-cinderclient | 13:22 |
yoctozepto | python-keystoneclient | 13:22 |
*** jimcrowleyibm has joined #openstack-kolla | 13:22 | |
yoctozepto | python-glanceclient | 13:22 |
yoctozepto | so 4 pins more | 13:22 |
yoctozepto | I'll create a bug report | 13:22 |
yoctozepto | so that you can fix it | 13:23 |
*** noxoid has joined #openstack-kolla | 13:24 | |
yoctozepto | mgoddard: https://bugs.launchpad.net/kolla/+bug/1870349 go for it | 13:26 |
openstack | Launchpad bug 1870349 in kolla stein "unpinned deps in kolla-toolbox break due to py2 incompatibilities" [Critical,Triaged] | 13:26 |
*** markmcclain has joined #openstack-kolla | 13:26 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Install python-openstackclient using upper constraints https://review.opendev.org/715880 | 13:27 |
hrw | yoctozepto: https://review.opendev.org/716609 helps with it | 13:27 |
patchbot | patch 716609 - kolla - use proper Python version in virtualenv - 1 patch set | 13:27 |
hrw | a bit | 13:28 |
*** ykarel is now known as ykarel|afk | 13:29 | |
mgoddard | hrw: does it fix the issue? | 13:30 |
hrw | mgoddard: have to check. had other issues to solve | 13:30 |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Install python-openstackclient using upper constraints https://review.opendev.org/715880 | 13:31 |
mgoddard | hrw: I'd guess not - the problem is with the package we are installing, it does not say that it is py3 only | 13:31 |
hrw | ok | 13:31 |
openstackgerrit | Mark Goddard proposed openstack/kolla stable/stein: Pin openstacksdk and client libraries in kolla-toolbox https://review.opendev.org/717008 | 13:42 |
mgoddard | yoctozepto: ^ | 13:42 |
hrw | mgoddard: does train needs it too? | 13:43 |
mgoddard | hrw: in Train we use upper-constraints | 13:43 |
kplant | so... i've evacuated a compute node, turn off all the containers. ran a yum upgrade and rebooted | 13:47 |
kplant | now it looks like instances that i move back to this hypervisor can't pass L2 traffic on existing tenant networks | 13:48 |
kplant | any idea what i broke? | 13:48 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Separate per-service host configuration tasks https://review.opendev.org/703127 | 13:52 |
*** weshay is now known as weshay|ruck | 13:55 | |
*** vishalmanchanda has quit IRC | 13:58 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Introduce /etc/timezone to containers https://review.opendev.org/704258 | 14:04 |
*** TrevorV has joined #openstack-kolla | 14:07 | |
*** ktibi has quit IRC | 14:09 | |
yoctozepto | kplant: networking! what is the prize? | 14:10 |
*** ktibi has joined #openstack-kolla | 14:10 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: CentOS 8: seed VM & bifrost https://review.opendev.org/712133 | 14:10 |
kplant | yoctozepto: me punching myself in the face over and over again | 14:13 |
* yoctozepto completely satisfied | 14:16 | |
yoctozepto | kplant: did you check what ovs agent logs say? | 14:19 |
kplant | lmao | 14:19 |
yoctozepto | and whether they say anything at all, maybe it did not come up | 14:19 |
kplant | everything looks clean :-| | 14:20 |
openstackgerrit | Mark Goddard proposed openstack/kolla stable/rocky: Pin openstacksdk and client libraries in kolla-toolbox https://review.opendev.org/717024 | 14:20 |
yoctozepto | then it's fine and you are exaggerating! | 14:20 |
kplant | that's why i was asking about docker-ce 19 earlier | 14:20 |
kplant | maybe i went too far into the future | 14:20 |
kplant | i was living in the year 3020 | 14:20 |
yoctozepto | nah, it seems worky in our CI | 14:20 |
kplant | i think my cluster has coronavirus | 14:21 |
kplant | that's the only explanation | 14:21 |
yoctozepto | first case | 14:21 |
kplant | human to machine transmission | 14:21 |
*** pbing19 has quit IRC | 14:23 | |
*** pbing19 has joined #openstack-kolla | 14:26 | |
*** ykarel|afk is now known as ykarel | 14:28 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: Install python-openstackclient using upper constraints https://review.opendev.org/717031 | 14:28 |
kplant | i'll keep digging | 14:28 |
kplant | good to know you guys are having no issues with 19.x though | 14:28 |
*** abdysn has joined #openstack-kolla | 14:34 | |
*** Abdallahyas has quit IRC | 14:34 | |
yoctozepto | yeah, it's probably related to the upgrade/restart itself | 14:37 |
mgoddard | kplant: one thing that does bite from time to time is that docker introduced a default DROP policy on the FORWARD chain in the filter table of iptables | 14:37 |
kplant | ooooooooooof | 14:37 |
yoctozepto | but we mitigate that | 14:38 |
mgoddard | can't remember when they introduced it, but it's a PITA. So far I think it mostly affects development environments | 14:38 |
mgoddard | yoctozepto: what do you mean? | 14:38 |
yoctozepto | mgoddard: well, don't we? how come it works? :D | 14:38 |
mgoddard | yoctozepto: neutron iptables rules? | 14:39 |
yoctozepto | mgoddard: no idea, I even slapped a comment about it in code | 14:39 |
yoctozepto | pre.yml | 14:39 |
yoctozepto | # FIXME(yoctozepto): really handle this for users - somehow my local multinode | 14:39 |
yoctozepto | # deploy fixed it for itself by setting it to ACCEPT on network nodes without | 14:39 |
yoctozepto | # my intervention but so far we have no idea what did that. It certainly does | 14:39 |
yoctozepto | # not happen in CI where all nodes are aio. | 14:39 |
yoctozepto | I seriously did not introduce any own workaround ;p | 14:40 |
yoctozepto | though it was needed for CI itself | 14:40 |
yoctozepto | dubious topic | 14:40 |
*** gfidente has quit IRC | 14:45 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Install python-openstackclient using upper constraints https://review.opendev.org/715880 | 14:56 |
*** gfidente has joined #openstack-kolla | 14:58 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Install python-openstackclient using upper constraints https://review.opendev.org/715880 | 14:59 |
openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: Install python-openstackclient using upper constraints https://review.opendev.org/717031 | 14:59 |
openstackgerrit | Merged openstack/kolla-ansible master: Support setting Kafka storage volume https://review.opendev.org/715307 | 15:11 |
cosmicsound | req-5c19c629-78c8-40ac-8ac6-568eec9fb3baattach_volume2 Apr 2020, 9:36 a.m.7096c304e64b4d5a87fa4d7cb8808042Error | 15:13 |
cosmicsound | Anyone knows how to debug such issues | 15:14 |
cosmicsound | this is a master k8s machine and the logs throw me this | 15:14 |
*** dmellado has quit IRC | 15:14 | |
cosmicsound | yet the machine have local storage: Disk /dev/mapper/atomicos-root: 29 GiB, 31134318592 bytes, 60809216 sectors | 15:14 |
cosmicsound | Not sure whats the attachement related to. I run ceph | 15:15 |
*** sorin-mihai has joined #openstack-kolla | 15:20 | |
*** jimcrowleyibm has quit IRC | 15:20 | |
*** abdysn has quit IRC | 15:24 | |
yoctozepto | cosmicsound: what cinder-volume says? | 15:25 |
*** dmellado has joined #openstack-kolla | 15:30 | |
cosmicsound | nothing... | 15:32 |
cosmicsound | Not about these requests | 15:32 |
yoctozepto | mgoddard, mnasiadka; INFO:kolla.common.utils.kolla-toolbox:Cannot find a valid baseurl for repo: centos-nfs-ganesha28 | 15:34 |
yoctozepto | so it hit c7 as well. We need megafix again... | 15:34 |
cosmicsound | yoctozepto , here is what it says when I create a new k8s with magnum | 15:36 |
cosmicsound | http://paste.openstack.org/show/791544/ | 15:36 |
yoctozepto | mnasiadka: you were right about fubar, argh | 15:37 |
yoctozepto | cosmicsound: "completed successfully" sounds positive to me | 15:37 |
cosmicsound | yoctozepto , check this: http://paste.openstack.org/show/791545/ | 15:39 |
*** hjensas is now known as hjensas|afk | 15:39 | |
cosmicsound | https://mdb.uhlhost.net/uploads/6c866f75c9e0750e/image.png | 15:39 |
yoctozepto | cosmicsound: and volumes otherwise work? | 15:42 |
yoctozepto | cosmicsound: outside of magnum things | 15:42 |
cosmicsound | yes | 15:46 |
cosmicsound | even the machine with magnum has inside the volume | 15:47 |
cosmicsound | just that the error... | 15:47 |
*** ykarel is now known as ykarel|away | 15:47 | |
yoctozepto | cosmicsound: what backend is it? | 15:48 |
cosmicsound | ceph | 15:49 |
cosmicsound | yoctozepto , | 15:49 |
cosmicsound | cant this libvirt values be the issue? | 15:49 |
cosmicsound | https://mdb.uhlhost.net/uploads/ec1587d9f3e39358/image.png | 15:49 |
openstackgerrit | Merged openstack/kayobe master: Fix loading of Monasca Grafana dashboards https://review.opendev.org/715638 | 15:50 |
cosmicsound | they kinda relate to nova errors about scsi | 15:50 |
yoctozepto | cosmicsound: no, these are right for ceph in general | 15:53 |
cosmicsound | ok | 15:53 |
yoctozepto | you might also want to turn on quiesce for flexible snapshots | 15:53 |
yoctozepto | but that's orthogonal | 15:53 |
yoctozepto | see https://docs.ceph.com/docs/master/rbd/rbd-openstack/#image-properties | 15:53 |
yoctozepto | anyways, it's weird | 15:53 |
yoctozepto | you are telling me that doing nova instance and mounting an extra volume there works | 15:54 |
yoctozepto | but if magnum does it, then there is this ugly error | 15:54 |
cosmicsound | yes | 15:54 |
cosmicsound | true that | 15:54 |
cosmicsound | Can devicemapper or overlay play a role on this? | 15:55 |
yoctozepto | cosmicsound: I wanted to tell you to ask on #openstack-containers but you already did | 15:55 |
yoctozepto | cosmicsound: what about magnum logs? | 15:55 |
*** rpittau is now known as rpittau|afk | 15:56 | |
yoctozepto | maybe magnum tries to mount it before giving to nova? | 15:56 |
yoctozepto | we can continue this on #...containers, folks probably know how it works | 15:56 |
cosmicsound | ok | 15:56 |
cosmicsound | seems these reqs are only from nova | 15:57 |
*** bengates has quit IRC | 15:57 | |
cosmicsound | Nowhere i found the in whole /var/log | 15:57 |
mgoddard | yoctozepto: do you know what the centos-nfs-ganesha issue is? | 16:01 |
hrw | https://review.opendev.org/#/c/716609 - who adds +w? | 16:10 |
patchbot | patch 716609 - kolla - use proper Python version in virtualenv - 1 patch set | 16:10 |
openstackgerrit | Dincer Celik proposed openstack/kolla-ansible master: Introduce /etc/timezone to containers https://review.opendev.org/704258 | 16:10 |
hrw | and then we can backport it | 16:10 |
mgoddard | hrw: CI is broken | 16:10 |
hrw | meh | 16:12 |
hrw | let me run some runs | 16:15 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Use unittest.mock instead of PyPI mock https://review.opendev.org/717073 | 16:16 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla stable/stein: Pin openstacksdk and client libraries in kolla-toolbox https://review.opendev.org/717008 | 16:18 |
*** skramaja has quit IRC | 16:20 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Use unittest.mock instead of PyPI mock https://review.opendev.org/717073 | 16:21 |
hrw | hm... | 16:22 |
openstackgerrit | Mark Goddard proposed openstack/kolla stable/stein: Fix multiple CI issues https://review.opendev.org/717008 | 16:22 |
hrw | why we pin keystoneclient 3.18.0 when requirements pin 3.19.1 which is py2/3 on pypi? | 16:22 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Use unittest.mock instead of PyPI mock https://review.opendev.org/717075 | 16:23 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla stable/rocky: Pin openstacksdk and client libraries in kolla-toolbox https://review.opendev.org/717024 | 16:23 |
hrw | anyway fixed both patches | 16:23 |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Update documentation for release tasks https://review.opendev.org/709154 | 16:23 |
yoctozepto | hrw: dunno, I checked Mark pinned to what u-c pin in stable branches | 16:23 |
openstackgerrit | Mark Goddard proposed openstack/kolla stable/rocky: Fix multiple CI issues https://review.opendev.org/717024 | 16:24 |
hrw | I know why we did that way | 16:24 |
hrw | kolla-toolbox does not use constraints | 16:24 |
mgoddard | right | 16:24 |
mgoddard | I added an upper limit of the first version from the next release | 16:25 |
mgoddard | means we do not have to keep bumping for new releases | 16:25 |
yoctozepto | the mirror fix is gating now | 16:26 |
mgoddard | cool | 16:26 |
yoctozepto | hopefully gets systems reconfigured before monasca-grafana does last try | 16:26 |
cosmicsound | Important Using QCOW2 for hosting a virtual machine disk is NOT recommended. If you want to boot virtual machines in Ceph (ephemeral backend or boot from volume), please use the raw image format within Glance. | 16:27 |
cosmicsound | Maybe this helps... | 16:27 |
*** ktibi has quit IRC | 16:28 | |
hrw | mgoddard: looks like I have better fix ;D | 16:33 |
mgoddard | hrw: which one? | 16:34 |
openstackgerrit | Scott Solkhon proposed openstack/kolla-ansible master: Support Neutron policy file in both .json and .yaml format https://review.opendev.org/702682 | 16:34 |
hrw | just have to port it to master, check and send for review | 16:34 |
hrw | mgoddard: use constraints | 16:34 |
mgoddard | hrw: agreed with yoctozepto earlier not to do that | 16:34 |
mgoddard | we already use constraints in master & train | 16:34 |
hrw | so why not in earlier? | 16:35 |
mgoddard | but don't want to introduce it to a stable branch because it requires access to the internet to get the constraints file | 16:35 |
mgoddard | so if you can't access the internet it will break your build | 16:35 |
hrw | you need net access to build images | 16:36 |
mgoddard | nope | 16:36 |
mgoddard | shouldn't assume that, some don't | 16:36 |
hrw | then how do you build openstack-base? | 16:36 |
*** evrardjp has quit IRC | 16:36 | |
*** evrardjp has joined #openstack-kolla | 16:36 | |
hrw | I see... | 16:37 |
mgoddard | local mirrors | 16:38 |
hrw | my k-t fix do the same | 16:38 |
hrw | let me just test does it build first | 16:39 |
hrw | https://paste.centos.org/view/fa425d08 makes kolla-toolbox use requirements tarball (like openstack-base does) and use constraints for all pip | 16:42 |
hrw | no more curl from somewhere | 16:42 |
yoctozepto | hrw: the problem is 'binary' ;p | 16:45 |
hrw | abomination | 16:49 |
openstackgerrit | Merged openstack/kolla-ansible master: Avoid unconditional fact gathering https://review.opendev.org/716340 | 16:50 |
hrw | INFO:kolla.common.utils.kolla-toolbox:Step 1/17 : FROM ma/debian-binary-base:9.1.0 | 16:59 |
hrw | INFO:kolla.common.utils.kolla-toolbox:Step 7/17 : ADD kolla-toolbox-archive /kolla-toolbox-source | 16:59 |
hrw | d | 16:59 |
hrw | :D | 16:59 |
hrw | INFO:kolla.common.utils.kolla-toolbox:Successfully tagged ma/debian-binary-kolla-toolbox:9.1.0 | 16:59 |
yoctozepto | impossible :O | 16:59 |
hrw | but not a fan of it | 16:59 |
yoctozepto | then we have another bug :D | 16:59 |
yoctozepto | ah, you hacked it | 16:59 |
hrw | https://paste.centos.org/view/0dd89127 adds special treatment for k-t | 16:59 |
cosmicsound | I am thinking on dumping ubuntu | 17:03 |
cosmicsound | in favor of centos | 17:03 |
cosmicsound | or debian | 17:03 |
hrw | wise choice | 17:03 |
cosmicsound | is centos 8 supported in k-a yet? | 17:03 |
cosmicsound | ubuntu seems to be chasing their non profit business model ... blurps me | 17:04 |
* hrw prefers not to comment what ubuntu is chasing | 17:04 | |
hrw | pep8-- | 17:04 |
hrw | python-- | 17:05 |
hrw | splitting 'if this or that' into two lines is nightmare | 17:05 |
yoctozepto | use parens | 17:05 |
hrw | or \ | 17:05 |
hrw | if self.install_type == 'source' or \ | 17:06 |
hrw | image.name in SOURCES_EVEN_FOR_BINARY: | 17:06 |
yoctozepto | folks tell me it's bad to use \ | 17:06 |
yoctozepto | :D | 17:06 |
hrw | flake8 says OK | 17:06 |
yoctozepto | for readability | 17:06 |
yoctozepto | yeah, I don't mind either | 17:06 |
yoctozepto | unless new hacking does | 17:06 |
hrw | 80 columns of pep8 is bad for readability | 17:06 |
hrw | 120 should be new 80 | 17:07 |
yoctozepto | true that | 17:07 |
hrw | 19:07 (0s) hrw@puchatek:~$ echo $COLUMNS $LINES | 17:07 |
hrw | 154 57 | 17:07 |
* cosmicsound hrw good point, idea is simple them made me to find alteratives | 17:08 | |
hrw | https://paste.centos.org/view/20e18697 is nicer version | 17:09 |
hrw | open for other exceptions | 17:09 |
hrw | should just work in each supported branch | 17:10 |
hrw | with fuzz tweaks and archive name tweak of course | 17:10 |
openstackgerrit | Merged openstack/kayobe master: Switch to stackhpc fork of resmo.ntp https://review.opendev.org/715893 | 17:11 |
hrw | yoctozepto: opinion? | 17:12 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: kolla-toolbox: use requirements for pip constraints https://review.opendev.org/717080 | 17:15 |
hrw | whatever ;d | 17:16 |
hrw | I consider it 'do it once and forget about the problem forever' solution | 17:16 |
yoctozepto | hrw: :D | 17:26 |
yoctozepto | hrw: it makes sense to fix airgapped this way | 17:27 |
*** KeithMnemonic has joined #openstack-kolla | 17:28 | |
openstackgerrit | Will Szumski proposed openstack/kayobe stable/train: Switch to stackhpc fork of resmo.ntp https://review.opendev.org/717081 | 17:28 |
openstackgerrit | Will Szumski proposed openstack/kayobe stable/stein: Switch to stackhpc fork of resmo.ntp https://review.opendev.org/717082 | 17:29 |
cosmicsound | OSA had built in ceph support? | 17:33 |
cosmicsound | Or will follow same road lika k-a | 17:33 |
*** gfidente is now known as gfidente|afk | 17:39 | |
yoctozepto | cosmicsound: I doubt, I believe we were unique in that regard; but trust me - leave ceph to ceph ;-) | 17:40 |
*** ironfoot has quit IRC | 17:43 | |
mnasiadka | cosmicsound: OSA uses ceph-ansible, just has a minimal level of integration (imports ceph-ansible roles when deploying ceph) | 17:43 |
*** ironfoot has joined #openstack-kolla | 17:43 | |
*** k_mouza has quit IRC | 17:46 | |
*** jimcrowleyibm has joined #openstack-kolla | 17:52 | |
*** iniazi has joined #openstack-kolla | 18:03 | |
*** k_mouza has joined #openstack-kolla | 18:13 | |
cosmicsound | thanks mnasiadka , yoctozepto | 18:13 |
*** k_mouza has quit IRC | 18:14 | |
openstackgerrit | Dincer Celik proposed openstack/kolla-ansible master: Introduce /etc/timezone to containers https://review.opendev.org/704258 | 18:19 |
*** Tony31 has quit IRC | 18:28 | |
bline | yoctozepto: ubuntu train | 18:37 |
cosmicsound | Failed to delete image with name or ID '3482ed4d-571c-4a64-a8ab-07b0c45aae21': HTTP 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. | 18:49 |
cosmicsound | what | 18:49 |
*** diga has quit IRC | 18:51 | |
yoctozepto | bline: thanks, noted | 19:12 |
cosmicsound | unable to delete an image | 19:16 |
cosmicsound | this is first ttime :D | 19:16 |
mgoddard | yoctozepto: looks like monasca-grafana is still busted :( | 19:24 |
mgoddard | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_40d/717024/3/check/kolla-build-ubuntu-source/40d6e06/logs/build/000_FAILED_monasca-grafana.log | 19:24 |
*** Chaserjim has joined #openstack-kolla | 19:34 | |
yoctozepto | mgoddard: system reconfigured only recently | 19:49 |
yoctozepto | mgoddard: zuul had some real delays | 19:49 |
yoctozepto | mgoddard: otoh, ganesha is still busted in that stein change anyway | 19:49 |
yoctozepto | another day, another trial | 19:49 |
yoctozepto | good night | 19:50 |
bline | thanks yoctozepto, I can add more details to that ticket if it would help. | 19:50 |
openstackgerrit | Merged openstack/kayobe master: Install python-openstackclient using upper constraints https://review.opendev.org/715880 | 20:07 |
*** jimcrowleyibm has quit IRC | 20:29 | |
*** TrevorV has quit IRC | 20:37 | |
*** jbadiapa has quit IRC | 20:42 | |
*** mrunge_ has joined #openstack-kolla | 20:48 | |
*** mrunge has quit IRC | 20:49 | |
*** xingchao has joined #openstack-kolla | 20:54 | |
*** weshay|ruck is now known as weshay | 20:57 | |
*** igordc has joined #openstack-kolla | 21:05 | |
*** e0ne has quit IRC | 21:06 | |
*** k_mouza has joined #openstack-kolla | 22:15 | |
*** k_mouza has quit IRC | 22:19 | |
*** cah_link1 has joined #openstack-kolla | 22:21 | |
*** cah_link has quit IRC | 22:24 | |
*** cah_link1 is now known as cah_link | 22:24 | |
*** ktibi has joined #openstack-kolla | 22:24 | |
*** hrw has quit IRC | 22:37 | |
*** ktibi has quit IRC | 22:40 | |
*** openstackstatus has quit IRC | 22:56 | |
*** openstack has joined #openstack-kolla | 22:59 | |
*** ChanServ sets mode: +o openstack | 22:59 | |
*** hrw has joined #openstack-kolla | 23:29 | |
*** Tony31 has joined #openstack-kolla | 23:40 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!