Monday, 2020-04-20

« Sunday, 2020-04-19 Index Tuesday, 2020-04-21 »
*** brinzhang has joined #openstack-qa00:20
*** paras333 has quit IRC00:33
*** slaweq has joined #openstack-qa00:50
*** brinzhang has quit IRC00:58
*** brinzhang has joined #openstack-qa01:01
*** yamamoto has joined #openstack-qa01:26
*** brinzhang has quit IRC01:28
*** brinzhang has joined #openstack-qa01:29
*** yamamoto has quit IRC01:58
*** yamamoto has joined #openstack-qa01:58
*** yamamoto has quit IRC02:04
*** yamamoto has joined #openstack-qa02:18
*** ramishra has joined #openstack-qa02:51
*** jhesketh has joined #openstack-qa02:56
openstackgerritQitao proposed openstack/tempest master: Use unittest.mock instead of third party mock  https://review.opendev.org/72115403:04
*** ricolin has joined #openstack-qa03:48
*** yamamoto has quit IRC03:52
*** yamamoto has joined #openstack-qa04:02
*** evrardjp has quit IRC04:23
*** udesale has joined #openstack-qa04:38
*** paras333 has joined #openstack-qa04:43
*** paras333 has quit IRC04:47
*** paras333 has joined #openstack-qa04:52
*** paras333 has quit IRC04:56
*** iokiwi has quit IRC05:16
*** iokiwi has joined #openstack-qa05:17
*** gcheresh has joined #openstack-qa05:29
*** yamamoto has quit IRC05:35
*** evrardjp has joined #openstack-qa05:41
*** vishalmanchanda has joined #openstack-qa05:46
*** yamamoto has joined #openstack-qa05:55
*** nightmare_unreal has joined #openstack-qa06:32
*** ccamposr has joined #openstack-qa07:01
*** ccamposr__ has quit IRC07:03
*** rpittau|afk is now known as rpittau07:12
*** ralonsoh has joined #openstack-qa07:18
openstackgerritQitao proposed openstack/tempest master: Use unittest.mock instead of third party mock  https://review.opendev.org/72115407:19
openstackgerritAndreas Jaeger proposed openstack/tempest master: Use unittest.mock instead of third party mock  https://review.opendev.org/72102107:21
*** amodi has quit IRC07:45
*** udesale_ has joined #openstack-qa07:47
*** udesale has quit IRC07:50
*** lucasagomes has joined #openstack-qa07:51
openstackgerritLee Yarwood proposed openstack/tempest master: remote_client: Use blkid -L to find config drive by filesystem label  https://review.opendev.org/71086407:52
openstackgerritLee Yarwood proposed openstack/tempest master: zuul: Promote devstack-plugin-ceph-tempest-py3 to the check queue  https://review.opendev.org/71047707:52
*** jpena|off is now known as jpena07:55
*** ccamposr has quit IRC08:39
*** ccamposr has joined #openstack-qa08:44
*** paras333 has joined #openstack-qa08:46
*** paras333 has quit IRC08:50
*** paras333 has joined #openstack-qa08:52
*** vishalmanchanda has quit IRC08:56
*** paras333 has quit IRC08:56
*** tosky has joined #openstack-qa09:01
*** ramishra has quit IRC09:02
openstackgerritzhufl proposed openstack/tempest master: Adding description for testcases - volume part2  https://review.opendev.org/72120809:10
*** ramishra has joined #openstack-qa09:11
*** dtantsur|afk is now known as dtantsur09:21
*** gregwork has quit IRC09:24
*** rcernin has quit IRC09:26
*** ricolin has quit IRC09:28
*** ricolin has joined #openstack-qa09:48
*** ricolin_ has joined #openstack-qa09:48
*** yamamoto has quit IRC09:49
*** rcernin has joined #openstack-qa09:51
*** vishalmanchanda has joined #openstack-qa09:53
*** gfidente has joined #openstack-qa09:55
openstackgerritBharat Kunwar proposed openstack/devstack master: Support upload of qcow2.xz image format  https://review.opendev.org/72121309:56
openstackgerritBharat Kunwar proposed openstack/devstack master: Support upload of qcow2.xz image format  https://review.opendev.org/72121309:59
*** ricolin_ has quit IRC10:02
*** yamamoto has joined #openstack-qa10:06
*** rpittau is now known as rpittau|bbl10:15
*** yamamoto has quit IRC10:34
*** yamamoto has joined #openstack-qa10:43
*** yamamoto has quit IRC10:49
*** yamamoto has joined #openstack-qa10:50
*** yamamoto has quit IRC10:50
*** paras333 has joined #openstack-qa10:52
*** paras333 has quit IRC10:56
*** yamamoto has joined #openstack-qa11:19
*** jpena is now known as jpena|lunch11:27
*** paras333 has joined #openstack-qa11:31
*** paras333 has quit IRC11:36
*** rh-jelabarre has joined #openstack-qa11:50
*** raildo has joined #openstack-qa11:51
openstackgerritAndreas Jaeger proposed openstack/tempest master: Use unittest.mock instead of third party mock  https://review.opendev.org/72102111:57
*** yamamoto has quit IRC11:57
openstackgerritAndreas Jaeger proposed openstack/tempest master: Use unittest.mock instead of third party mock  https://review.opendev.org/72102111:58
*** rpittau|bbl is now known as rpittau12:03
*** rfolco has joined #openstack-qa12:22
*** yamamoto has joined #openstack-qa12:30
*** jpena|lunch is now known as jpena12:33
*** yamamoto has quit IRC12:33
*** yamamoto has joined #openstack-qa12:33
*** udesale_ has quit IRC12:34
*** Luzi has joined #openstack-qa12:42
*** artom has joined #openstack-qa13:00
*** irclogbot_1 has quit IRC13:06
*** irclogbot_1 has joined #openstack-qa13:08
kopecmartinslaweq: when you have a moment, can you have a look at this please https://bugs.launchpad.net/tripleo/+bug/1852770  any idea what can be the root cause?13:13
openstackLaunchpad bug 1852770 in tripleo "No more IP addresses available on network, neutron_tempest_plugin.scenario.test_security_groups.NetworkSecGroupTest" [High,In progress] - Assigned to wes hayutin (weshayutin)13:13
slaweqkopecmartin: hi, ok I will check it13:15
slaweqkopecmartin: but now I'm on the meetings, I will probably be able to check it on evening13:16
kopecmartinslaweq: sure, not problem .. thank you13:16
*** paras333 has joined #openstack-qa13:24
slaweqkopecmartin: ok, I finished meeting a bit earlier so I commented it already13:26
slaweqkopecmartin: please let me know if You will need anything else regarding this issue13:26
kopecmartinslaweq: perfect, thanks!13:29
brtknrtosky: AJaeger: would it be okay to add support for qcow2.xz please? https://review.opendev.org/#/c/72126513:39
brtknrgmann: ^13:39
toskybrtknr: in the magnum devstack plugin? I guess that's up to you, or do you need any change in devstack core?13:41
brtknrtosky: yes, change in devstack core is required because magnum relies on IMAGE_URLS to upload the image13:42
brtknrbut devstack-core doesnt currently know how to handle qcow2.xz13:42
brtknrapparently glance doesnt either13:42
brtknrso makes sense to add support in devstack-core scope13:42
toskyuhm, it shouldn't be difficult to add it; basically devstack uncompress the image, so glance does not really need any change13:43
*** yamamoto has quit IRC13:44
toskyit's in the second part of upload_image of functions13:44
toskysee https://opendev.org/openstack/devstack/src/branch/master/functions#L29613:44
toskybasically extend the *.qcow2 case to handle *.qcow2.xz too, like vhd does (*.vhd|*.vhdx|*.vhd.gz|*.vhdx.gz)13:45
brtknrtosky: i have already imlemented the required change here:  https://review.opendev.org/#/c/72126513:45
brtknrjust need +2 :D13:45
brtknrtosky: oops sorry wrong link https://review.opendev.org/#/c/72121313:46
toskyoooh13:46
toskyI can +1 (I don't have +2)13:46
brtknrsweet thanks13:46
toskysomeone in the future may want to unify the 3 if [ "$unpack" = "<foo>" ]13:47
toskybut that's not important13:47
*** yamamoto has joined #openstack-qa13:48
*** yamamoto has quit IRC13:48
toskywould it make sense to support img.xz too?13:48
brtknrtosky: maybe but we wouldnt know what disk_format it was13:49
*** yamamoto has joined #openstack-qa13:49
brtknrwith qcow2.xz, its clear that the disk format is qcow213:49
brtknrtosky: i followed the same patttern as for img.gz13:50
brtknrand img.bz213:50
toskyinteresting enough, the current code a) checks for the proper format for .img files b) assumes it's raw for img.gz C) assumes it's qcow2 for img.bz213:51
toskywhat could possibly go wrong?13:51
toskyI suspect that 3 img* cases should be unified like the vhd ones; bug again, that's a different change13:52
openstackgerritBharat Kunwar proposed openstack/devstack master: Support upload of qcow2.xz image format  https://review.opendev.org/72121313:52
brtknrI have added img.xz13:53
brtknrit is odd that img.bz is qcow2 and13:53
toskyyeah13:54
brtknris img usually raw?13:54
toskyso maybe (and sorry for that) better just to add qcow2.xz in your change13:54
toskyand let's think more about that13:54
toskybut let's see what the cores think13:54
openstackgerritBharat Kunwar proposed openstack/devstack master: Support upload of qcow2.xz image format  https://review.opendev.org/72121313:55
brtknrok reverted13:55
-openstackstatus- NOTICE: Zuul is temporarily offline; service should be restored in about 15 minutes.13:59
toskythanks; sorry for the confusion14:01
*** hongbin has joined #openstack-qa14:21
gmannAJaeger: commented on this. TEMPEST_BRANCH var will work here -  https://review.opendev.org/#/c/721060/414:39
openstackgerritMerged openstack/tempest-stress master: Cleanup py27 support  https://review.opendev.org/71941814:39
*** weaselkeeper has joined #openstack-qa14:40
AJaegergmann: Ah, thanks!14:40
gmannkopecmartin: need to merge this also before we cut ussuri final release- https://review.opendev.org/#/c/721110/14:44
openstackgerritMerged openstack/tempest-stress master: Add publish-openstack-docs-pti to .zuul.yaml  https://review.opendev.org/72112614:46
brtknrtosky: i dont understand what you said here: https://review.opendev.org/#/c/720216/15/.zuul.yaml@10214:47
brtknrAre you saying i can just use name:magnum-tempest-plugin-tests-api14:47
brtknrtosky: ah I think i know what you mean14:48
toskybrtknr: you don't need to define a custom job entry; just use magnum-tempest-plugin-tests-api in the check.jobs and gate.jobs entries14:48
brtknryep gotcha :)14:48
toskydo you have other jobs which use those playbooks/? If not, you may remove them in that same review14:50
*** gcheresh has quit IRC14:50
kopecmartingmann: masayukig, when you have a moment, can you check this?14:55
kopecmartingmann: masayukig https://review.opendev.org/#/c/701434/14:55
kopecmartingmann: masayukig i had one question there i can't find an answer to14:56
*** rh-jelabarre has quit IRC14:58
brtknrtosky: done14:59
*** gcheresh has joined #openstack-qa15:06
*** dtroyer has joined #openstack-qa15:07
*** rh-jelabarre has joined #openstack-qa15:09
*** vishalmanchanda has quit IRC15:26
*** hongbin has quit IRC15:28
*** amodi has joined #openstack-qa15:46
-openstackstatus- NOTICE: Gerrit will be restarted to correct a misconfiguration which caused some git mirrors to have outdated references.15:47
*** hongbin has joined #openstack-qa16:05
*** rpittau is now known as rpittau|afk16:07
*** lucasagomes has quit IRC16:09
*** Luzi has quit IRC16:15
*** dtantsur is now known as dtantsur|afk16:32
*** evrardjp has quit IRC16:35
*** evrardjp has joined #openstack-qa16:35
openstackgerritGhanshyam Mann proposed openstack/tempest master: Add releasenote to tag the Tempest for Ussuri release  https://review.opendev.org/72130016:40
openstackgerritGhanshyam Mann proposed openstack/patrole master: Add releasenote to tag the Patrole for Ussuri release  https://review.opendev.org/72130516:50
*** gfidente is now known as gfidente|afk16:53
*** csatari has quit IRC16:54
*** kklimonda has quit IRC16:54
*** kklimonda has joined #openstack-qa16:55
*** csatari has joined #openstack-qa16:56
gmannmasayukig: kopecmartin proposed the releasenotes patch -   https://review.opendev.org/#/c/721300/17:01
openstackgerritGhanshyam Mann proposed openstack/tempest-stress master: [ussuri][goal] Update contributor documentation  https://review.opendev.org/72112017:10
openstackgerritMerged openstack/tempest master: Fix noVNC failed TCs  https://review.opendev.org/71778617:18
*** ricolin has quit IRC17:29
*** jpena is now known as jpena|off17:32
openstackgerritMerged openstack/tempest master: Catch json.loads exceptions in verify_device_metadata  https://review.opendev.org/71361217:36
openstackgerritMerged openstack/tempest master: [ussuri][goal] Update contributor documentation  https://review.opendev.org/72111017:36
*** ralonsoh has quit IRC17:39
*** evrardjp has quit IRC17:44
*** evrardjp has joined #openstack-qa17:49
*** ccamposr__ has joined #openstack-qa17:50
*** paras333 has quit IRC17:51
*** ccamposr has quit IRC17:52
*** paras333 has joined #openstack-qa17:55
AJaegergmann: is my change for heat-tempest-plugin wrong? Do you have time to advise again, please?  https://review.opendev.org/72106017:57
*** nightmare_unreal has quit IRC18:02
*** hongbin has quit IRC18:21
*** hongbin has joined #openstack-qa18:26
*** hongbin has quit IRC18:27
*** slaweq has quit IRC18:50
*** slaweq has joined #openstack-qa18:56
*** slaweq_ has joined #openstack-qa19:02
*** slaweq has quit IRC19:05
*** gcheresh has quit IRC19:39
brtknrgmann: any idea why I’m getting image upload failure with glance? I don’t get the same issue locally. Looks like swift backend doesn’t like it20:04
clarkbbrtknr: is it too big for the swift install maybe?20:10
*** raildo_ has joined #openstack-qa21:07
*** raildo_ has quit IRC21:08
*** raildo has quit IRC21:10
brtknrmaybe?21:20
brtknrclarkb: what is the solution? disable swift in the ci?21:20
brtknrthis is only on the CI?21:20
clarkbbrtknr: I would start by confirming what the problem is21:21
clarkbhave you looked at the glance and swift logs to see why it fails?21:21
brtknrclarkb: https://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e21:21
brtknrin glance logs:21:21
brtknrhttps://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/screen-g-api.txt#108921:22
brtknrhttps://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/screen-g-api.txt#101221:23
brtknrclarkb: does that make any sense to you?21:24
brtknrnot me21:25
brtknrclarkb: https://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/screen-s-proxy.txt#20121:26
brtknrin proxy - i see 60s timeout21:26
clarkbbrtknr: https://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/apache/access_log.txt that apache log and maybe syslog might have clues to why there is a read timeout21:27
clarkbbut ya its definitely failing to uploda the image there it seems due to a currently unexplained read timeout21:27
brtknrclarkb: i thought it might be because image is too large?21:31
brtknrclarkb: uncompressed it is 1.7G21:31
brtknrcompressed 460M21:31
clarkbthat was an assumption of mine just thinking about ways it could fail. I don't have proof of that21:32
clarkba client read failure implies that isn't the issue though as that says the client isn't providing the bits quickly enough?21:32
clarkbI'm not a swift or a glance expert. timburke might be able to help on the swift side21:32
brtknrclarkb: i thnk it is because of this: https://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/etc/swift/proxy-server_conf.txt#8821:33
brtknrtlient timeout is 60s according to swift config21:33
clarkbright but why is there no bits flowing for 60 seconds21:33
brtknrany idea how i can modify swift config on the ci?21:33
clarkbthe timeout is there to catch the problems early21:33
clarkbmodifying the config may not help21:33
brtknrclarkb: because it is uncompressing the image?21:34
clarkbis that because glance will decompress a compressed image?21:35
clarkbI guess that is possible21:35
brtknrclarkb: this is from the glance api: https://zuul.opendev.org/t/openstack/build/b523eaca9dd24b9b9c98260f5752e09e/log/controller/logs/screen-g-api.txt21:39
brtknroops sorry i forgot i already shared that21:39
*** paras333 has quit IRC21:42
*** paras333 has joined #openstack-qa21:45
*** dosaboy has quit IRC21:46
brtknrclarkb: its stuck in the same place: http://zuul.openstack.org/stream/a8657839baca4fb79bd7f5fe50d91907?logfile=console.log21:54
brtknrfor the past 5 mins21:54
clarkbbrtknr: one thing that might be worth looking into is having glance write chunks of decompressed data more frequently to avoid swift timeouts21:55
clarkbif that is the issue then rather than decompress the whole thing and write it like that. Do the decompression "block" by "block" and write the results21:56
brtknrthat is why the input is being piped in21:58
brtknrclarkb: openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create fedora-coreos-31.20200323.3.2-openstack.x86_64 --property hw_rng_model=virtio --public --container-format=bare --disk-format qcow2 < <(unxz -c image.qcow2.xz)21:58
brtknrthat is the command that gets executed21:58
clarkbbrtknr: I want to say xz may take its time before writing anything22:01
clarkbthough I don't use xz often to know for sure. Maybe try a separate unxz step then upload the result22:01
clarkbqcow2 is already compressed so double compression may hurt~22:01
clarkb?22:01
*** dosaboy has joined #openstack-qa22:02
*** rcernin has quit IRC22:05
*** rcernin has joined #openstack-qa22:06
brtknrclarkb: yay it worked after disabling swift but dear god, it takes 10 mins22:10
clarkbbrtknr: its also not fixing the problem really22:11
clarkbits just using a different system instead22:11
clarkband yes xz is expensive22:11
brtknrjust to uncompress and upload22:11
brtknrunxz = decompress image22:11
clarkbyup22:11
brtknrnot sure if there is another way around that...22:12
clarkbyou can stop using xz22:12
brtknrclarkb: we cant, that is how the image is distributed22:13
clarkbif that is how upstream is distributing the imgae then ya no way around that22:13
clarkbthe only other way to make it faster is faster cpus I think22:13
brtknrclarkb: can i ask for faster cpus in the ci?22:14
clarkbbrtknr: we don't relaly have contorl over that no22:15
brtknrclarkb: is it possible for a local cache to save uncompressed image somewhere?22:16
clarkbbrtknr: yes, we have mirrors for fedora that had been keeping atomic images. I think that may have gotten shuffled around with the coreos switch but you could update it to capture those images if not already there22:17
brtknrclarkb: so we are planning to stop using atomic images since is EOL, and switch over to coreos22:18
brtknrclarkb: so we are planning to stop using atomic images since is EOL, and switch over to fedora coreos22:18
clarkbhttp://mirror.ord.rax.opendev.org/fedora/atomic/stable/ yup that seems to have gone stale22:19
clarkbbrtknr: understood just pointing out we do the thing for atomic already. Its just a matter of someone updating it to do whatever is appropriate for coreos now22:19
*** tosky has quit IRC22:19
brtknris this someting i can do or have to ask someone?22:19
openstackgerritBharat Kunwar proposed openstack/devstack master: Support upload of qcow2.xz image format  https://review.opendev.org/72121322:33
*** mtreinish has joined #openstack-qa22:38
clarkbbrtknr: sorry been distracted by a few other things22:51
clarkbbrtknr: you can do it, let me see if I can find the existing script22:51
clarkbbrtknr: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/mirror-update/files/fedora-mirror-update#L94-L121 that is where we sync in the atomic images22:52
clarkbwe likely want similar for whatever is appopriate for the coreos images22:52
clarkbalso we should be careful that we don't oversync since disk space is limited. eg ppc images don't help us22:52
clarkbneither do having 5 different formats of the same image22:52
clarkbbrtknr: I don't know where to find fedora coreos imgae mirrors we can rsync from, if you do let me know and I can keep helping but I think that is the next step. Basically find out what a valid source is23:01
*** slaweq_ has quit IRC23:21
*** slaweq_ has joined #openstack-qa23:25
*** slaweq_ has quit IRC23:29
masayukigkopecmartin: gmann: I put a comment. thoughts? https://review.opendev.org/#/c/701434/23:30
gmannmasayukig: kopecmartin checking23:50
gmannmasayukig: kopecmartin replied. we can do that as part of scenairo manager work.23:56
gmannkopecmartin: soniya29 can one of you add this item also for scenario manager etherpad - https://review.opendev.org/#/c/701434/12/tempest/api/compute/base.py@59523:57
gmannAJaeger: seems that is failing form heat side installation. I will debug it tomorrow.23:57
« Sunday, 2020-04-19 Index Tuesday, 2020-04-21 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!