*** tetsuro has joined #openstack-infra | 00:04 | |
*** armax has quit IRC | 00:29 | |
*** xavpaice has quit IRC | 00:31 | |
*** xavpaice has joined #openstack-infra | 00:34 | |
*** ramishra has joined #openstack-infra | 02:04 | |
*** zzzeek has quit IRC | 02:20 | |
*** zzzeek has joined #openstack-infra | 02:22 | |
*** tetsuro has quit IRC | 02:48 | |
*** psachin has joined #openstack-infra | 03:37 | |
*** zzzeek has quit IRC | 03:50 | |
*** zzzeek has joined #openstack-infra | 03:51 | |
*** ykarel has joined #openstack-infra | 04:32 | |
*** evrardjp has quit IRC | 04:33 | |
*** evrardjp has joined #openstack-infra | 04:33 | |
*** ramishra has quit IRC | 04:41 | |
*** vishalmanchanda has joined #openstack-infra | 04:45 | |
*** lmiccini has joined #openstack-infra | 05:51 | |
*** matt_kosut has joined #openstack-infra | 05:57 | |
*** lpetrut has joined #openstack-infra | 05:57 | |
*** ralonsoh has joined #openstack-infra | 06:05 | |
*** ysandeep|away is now known as ysandeep | 06:10 | |
*** fresta has joined #openstack-infra | 06:12 | |
*** cgoncalves has joined #openstack-infra | 06:13 | |
*** ramishra has joined #openstack-infra | 06:23 | |
*** eolivare has joined #openstack-infra | 06:31 | |
openstackgerrit | Carlos Goncalves proposed openstack/project-config master: Update branch checkout for octavia-lib DIB element https://review.opendev.org/745877 | 06:33 |
---|---|---|
*** hashar has joined #openstack-infra | 06:40 | |
*** slaweq_ has joined #openstack-infra | 06:50 | |
*** slaweq_ is now known as slaweq | 06:56 | |
*** Ajohn has quit IRC | 06:56 | |
*** jtomasek has joined #openstack-infra | 06:58 | |
*** slaweq_ has joined #openstack-infra | 07:14 | |
*** slaweq has quit IRC | 07:15 | |
*** jpena|off is now known as jpena | 07:22 | |
*** jcapitao has joined #openstack-infra | 07:29 | |
*** jtomasek has quit IRC | 07:39 | |
*** jtomasek has joined #openstack-infra | 07:41 | |
*** andrewbonney has joined #openstack-infra | 07:42 | |
*** gfidente has joined #openstack-infra | 07:42 | |
*** ysandeep is now known as ysandeep|lunch | 07:46 | |
*** tosky has joined #openstack-infra | 08:27 | |
*** derekh has joined #openstack-infra | 08:42 | |
*** lucasagomes has joined #openstack-infra | 08:55 | |
*** ysandeep|lunch is now known as ysandeep | 09:10 | |
*** sshnaidm|pto is now known as sshnaidm | 09:10 | |
*** xavpaice has quit IRC | 09:45 | |
*** howell has quit IRC | 09:45 | |
*** xavpaice has joined #openstack-infra | 09:46 | |
*** howell has joined #openstack-infra | 09:46 | |
*** xek has joined #openstack-infra | 09:47 | |
*** xek has quit IRC | 09:54 | |
*** smarcet has joined #openstack-infra | 10:04 | |
*** slaweq_ is now known as slaweq | 10:12 | |
*** e0ne has joined #openstack-infra | 10:24 | |
*** dtantsur|afk is now known as dtantsur | 10:27 | |
*** Lucas_Gray has joined #openstack-infra | 10:30 | |
*** jcapitao is now known as jcapitao_lunch | 10:34 | |
e0ne | hi. could anybody please help with vitrage gates troubleshooting? we've got issues: controller | xz: (stdin): Cannot allocate memory | 10:42 |
*** Lucas_Gray has quit IRC | 10:42 | |
e0ne | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_631/751657/2/check/vitrage-tempest-plugin-datasources/6316f3f/job-output.txt | 10:42 |
*** Lucas_Gray has joined #openstack-infra | 10:47 | |
*** Lucas_Gray has quit IRC | 10:51 | |
*** Wryhder has joined #openstack-infra | 10:51 | |
*** Wryhder is now known as Lucas_Gray | 10:51 | |
*** jpena is now known as jpena|lunch | 11:31 | |
*** dave-mccowan has joined #openstack-infra | 11:48 | |
*** ramishra has quit IRC | 11:48 | |
*** Lucas_Gray has quit IRC | 11:50 | |
*** ramishra has joined #openstack-infra | 11:52 | |
*** Lucas_Gray has joined #openstack-infra | 11:53 | |
*** rfolco has joined #openstack-infra | 11:55 | |
*** rfolco is now known as rfolco|ruck | 11:55 | |
*** Lucas_Gray has quit IRC | 11:58 | |
*** jcapitao_lunch is now known as jcapitao | 11:59 | |
*** lpetrut has quit IRC | 12:00 | |
*** lpetrut has joined #openstack-infra | 12:01 | |
zbr | interesting thread on pbr: https://github.com/realpython/python-guide/issues/494#event-3762763128 | 12:03 |
*** ramishra_ has joined #openstack-infra | 12:05 | |
*** smarcet has quit IRC | 12:08 | |
*** ramishra has quit IRC | 12:08 | |
*** eharney has joined #openstack-infra | 12:10 | |
*** smarcet has joined #openstack-infra | 12:10 | |
*** Goneri has joined #openstack-infra | 12:11 | |
*** rlandy has joined #openstack-infra | 12:14 | |
*** priteau has joined #openstack-infra | 12:14 | |
*** jpena|lunch is now known as jpena | 12:28 | |
*** vishalmanchanda has quit IRC | 12:28 | |
EmilienM | hey folks, codesearch seems down to me (fail to load) | 12:34 |
*** slaweq_ has joined #openstack-infra | 12:37 | |
*** xek has joined #openstack-infra | 13:00 | |
*** smarcet has quit IRC | 13:03 | |
zbr | clarkb: ianw: please look at https://review.opendev.org/#/c/751608/1 about install-from-bindep | 13:03 |
*** Lucas_Gray has joined #openstack-infra | 13:04 | |
openstackgerrit | Thierry Carrez proposed openstack/ptgbot master: Make clickable badges more button-like https://review.opendev.org/751781 | 13:05 |
openstackgerrit | Thierry Carrez proposed openstack/ptgbot master: Mark current time on the schedule https://review.opendev.org/751782 | 13:05 |
*** zxiiro has joined #openstack-infra | 13:05 | |
*** irclogbot_3 has quit IRC | 13:19 | |
*** irclogbot_2 has joined #openstack-infra | 13:25 | |
*** kaisers has joined #openstack-infra | 13:30 | |
*** artom has joined #openstack-infra | 13:33 | |
*** tkajinam has quit IRC | 13:37 | |
*** slaweq_ has quit IRC | 13:38 | |
*** jtomasek has quit IRC | 13:39 | |
*** ricolin has quit IRC | 13:52 | |
*** d34dh0r53 has joined #openstack-infra | 13:54 | |
*** smarcet has joined #openstack-infra | 14:01 | |
fungi | EmilienM: thanks for the heads up, it should be returning to service shortly (it had crashed with an odd json encoding panic) | 14:01 |
openstackgerrit | Merged openstack/ptgbot master: Make clickable badges more button-like https://review.opendev.org/751781 | 14:02 |
EmilienM | fungi: thanks! | 14:02 |
*** auristor has quit IRC | 14:05 | |
openstackgerrit | Merged openstack/ptgbot master: Mark current time on the schedule https://review.opendev.org/751782 | 14:05 |
*** sshnaidm is now known as sshnaidm|afk | 14:07 | |
*** ricolin has joined #openstack-infra | 14:11 | |
*** hashar has quit IRC | 14:14 | |
*** auristor has joined #openstack-infra | 14:18 | |
*** smarcet has quit IRC | 14:20 | |
*** smarcet has joined #openstack-infra | 14:23 | |
*** ykarel_ has joined #openstack-infra | 14:25 | |
*** ykarel has quit IRC | 14:28 | |
*** dklyle has joined #openstack-infra | 14:30 | |
*** ykarel__ has joined #openstack-infra | 14:32 | |
*** ykarel_ has quit IRC | 14:35 | |
*** ykarel__ is now known as ykarel | 14:35 | |
openstackgerrit | Carlos Goncalves proposed openstack/project-config master: Add 'check arm64' trigger to check-arm64 pipeline https://review.opendev.org/751829 | 14:36 |
*** slaweq_ has joined #openstack-infra | 14:41 | |
*** icey has quit IRC | 14:48 | |
*** icey has joined #openstack-infra | 14:49 | |
*** __ministry1 has joined #openstack-infra | 14:54 | |
*** slaweq_ has quit IRC | 14:55 | |
*** ykarel is now known as ykarel|away | 15:00 | |
*** lpetrut has quit IRC | 15:06 | |
*** armax has joined #openstack-infra | 15:12 | |
*** ykarel|away has quit IRC | 15:20 | |
noonedeadpunk | \o/ | 15:22 |
noonedeadpunk | have pretty nasty thing here and not sure whom I can ask for some help... We have issue with releasenotes, that they got merged for train and ussuri. I mean that reno for train is ok, but U ones have both train and ussuir | 15:23 |
noonedeadpunk | I'm not sure how reno distinguish them by releases.... | 15:23 |
*** Topner has joined #openstack-infra | 15:23 | |
clarkb | noonedeadpunk: I feel like I keep getting this wrong but it is my understanding that reno should run from a single context view that sees all branches and constructs the release notes. For everything in one document | 15:24 |
AJaeger | noonedeadpunk: best ask on openstack-release channel | 15:24 |
*** ianychoi has joined #openstack-infra | 15:24 | |
noonedeadpunk | ok, thanks! | 15:24 |
fungi | yeah, the release team has the best handle on nuances of reno | 15:24 |
noonedeadpunk | yeah, seems like this for me as well.... | 15:24 |
clarkb | zbr: can you explain why that change is necessary? I think the current expectation is that builds use the /output dir | 15:25 |
clarkb | zbr: I'm not sure I understand the problem that is trying to solve, you can write the image builds to use that dir | 15:25 |
*** lpetrut has joined #openstack-infra | 15:28 | |
*** xek has quit IRC | 15:35 | |
e0ne | hi. could anybody please help with vitrage gates troubleshooting? we've got issues: controller | xz: (stdin): Cannot allocate memory | 15:36 |
e0ne | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_631/751657/2/check/vitrage-tempest-plugin-datasources/6316f3f/job-output.txt | 15:36 |
clarkb | e0ne: that isn't the error causing the job to fail | 15:37 |
*** lmiccini has quit IRC | 15:38 | |
clarkb | e0ne: it looks like tox has 13 failed tests which is why the job fails. Then when it tries tocollect logs xz is unable to compress the journal due to a lack of memory (perhaps the journal is too big or there is very little memory, both may be related to the tox failure) | 15:38 |
clarkb | e0ne: it looks like we aren't getting other service logs due to the xz error, you may want to push a change up to devstack to make that do its best but if it fails the play should continue | 15:40 |
clarkb | (then you should get logs for the other services assuming the memory issue isn't critical) | 15:40 |
e0ne | clarkb: sorry, I didn't what are you proposing to do with a devstack | 15:43 |
clarkb | e0ne: make the export tasks in https://opendev.org/openstack/devstack/src/branch/master/roles/export-devstack-journal/tasks/main.yaml not fail on error | 15:44 |
clarkb | I think you can use ansible's failed_when: false for that | 15:44 |
e0ne | clarkb: will try it. thanks' | 15:45 |
clarkb | the idea is it should do its absolute best to get those logs but if there are problems (as in your case) then just continue and get whatever logs it can get | 15:45 |
e0ne | got it | 15:47 |
*** e0ne has quit IRC | 15:53 | |
*** zzzeek has quit IRC | 15:54 | |
*** zzzeek has joined #openstack-infra | 15:55 | |
*** lucasagomes has quit IRC | 16:01 | |
*** zzzeek has quit IRC | 16:02 | |
*** zzzeek has joined #openstack-infra | 16:04 | |
*** priteau has quit IRC | 16:04 | |
zbr | clarkb: re /output dir, at least for the e-r this folder contains only the files to be installed, there is no use for putting the wheels inside the builded containers | 16:09 |
zbr | is like including rpm/dep files inside container | 16:09 |
*** zzzeek has quit IRC | 16:09 | |
*** psachin has quit IRC | 16:09 | |
zbr | you want to install them, but you do not want to copy them inside the container, not into a layer. | 16:09 |
*** zzzeek has joined #openstack-infra | 16:10 | |
zbr | if you use dive tool to look inside the builder container you realise that output is likely not something we want. | 16:10 |
zbr | one trick that can be used to avoit this is to use /tmp instead, as that one is not stored inside. | 16:11 |
clarkb | zbr: the builder container is not part of the final product | 16:11 |
zbr | still, the current install-from-bindep is hardcoded to use /output | 16:11 |
*** jcapitao has quit IRC | 16:11 | |
clarkb | zbr: the builder container is a temporary throwaway scratch space so that we detroy all the unneeded build artifacts | 16:11 |
clarkb | zbr: it is meant to be used in conjunction with the python-base image which does form the basis of the final published images | 16:12 |
zbr | take a look at https://review.opendev.org/#/c/750958/8/Dockerfile | 16:12 |
zbr | like 25 does copy output from builder to final one, something we do not want. | 16:12 |
zbr | we do not need to store wheels on target one, we want to install them | 16:13 |
clarkb | yes please see how other images do this: https://opendev.org/zuul/zuul/src/branch/master/Dockerfile#L53-L55 I don't believe this is an issue | 16:13 |
zbr | i used dive tool to investigate containers and I seen layers with /output in them, not small at all. | 16:14 |
clarkb | zbr: did you look at projects like zuul which seem to do the right thing? | 16:15 |
*** zzzeek has quit IRC | 16:15 | |
zbr | i do not see a difference | 16:15 |
zbr | but i can try to investigate zuul image, to see if there is something different or not | 16:15 |
clarkb | Zuul intentionally rm's the /output dir when it is done with it | 16:15 |
clarkb | other images may not, but I think what zuul does is the expected use of the interface | 16:16 |
clarkb | if zuul looks good then my hunch is the problem is in the other consumers of python-builder not in the api presented by python-builder | 16:17 |
clarkb | (and we can fix that up/document it as necessary) | 16:17 |
* zbr looking at zuul.... | 16:18 | |
zbr | but as far as I know, the less layers the better. | 16:18 |
clarkb | right that should be one layer though because its all part of the single directive | 16:19 |
clarkb | basicaly it shouldn't add any more layers or data aiui | 16:19 |
zbr | with docker one can so something like: RUN --mount=type=bind,target=/output,from=elastic-recheck-builder,source=/output,rw command | 16:19 |
*** zzzeek has joined #openstack-infra | 16:19 | |
clarkb | ya that might be a cleaner way to express what is going on there | 16:20 |
zbr | this basically mounts the source, avoiding the copy operation. bad part, podman chokes on that, also old docker probably too. | 16:20 |
clarkb | but that still doesn't require a change to python-builder, just better use of the existing interfaces | 16:20 |
clarkb | ah not sure how important that is for projects like zuul but other places it is probably ok (like opendev images since we don't use podman) | 16:21 |
zbr | if the installer would allow being run with arbitrary folders, we could avoid this, w/o having to rely on features that are not available everywhere. | 16:22 |
zbr | "/output" was designed to be temporary, why it was not "/tmp/output"? | 16:22 |
*** derekh has quit IRC | 16:22 | |
clarkb | zbr: but why does it need to be arbitrary? I think that is my biggest concern here. Lots of code assumes /output and your change doesn't address any of that. If the goal is explicitly to use /tmp then maybe we should just use that instead (but I'm not sure how using /tmp works in this context if it is temporary since hte python-builder container is not running when python-base copies from it right?) | 16:23 |
*** derekh has joined #openstack-infra | 16:23 | |
clarkb | using arbitrary dirs and ensuring minimal image sizes are different goals | 16:23 |
zbr | the issue is the installer, if you try to run it from onether folder it chokes as it has hardcoded path inside. | 16:24 |
clarkb | the issue isn't hardcoded paths though right? the actual issue is you want to minimize image sizes? I'm saying we can hardcode /tmp instead and achieve the same goal | 16:25 |
clarkb | basically I want to understand the actual issue and address it rather thn just adding features to the tool that then allow people to continue to get things wrong if indeed we are doing things wrong in the zuul case | 16:25 |
clarkb | making thing sarbitrary makes that more difficult not easier | 16:25 |
zbr | depends how you look at it, IMHO the bug is the hardcoded /output in installer. If that would not be hardcoded, I could even use the install from multiple builders. | 16:26 |
clarkb | that is a different issue though. WHich is why I keep coming back to lets identify the actual issue and work from there | 16:26 |
clarkb | I think adding more flexibility here will make it easier for people to get minimal image sizes wrong | 16:27 |
clarkb | if the goal is to make minimal images then lets try to avoid adding too much flexibility | 16:27 |
*** __ministry1 has quit IRC | 16:27 | |
*** ralonsoh has quit IRC | 16:27 | |
zbr | but can we do this without breaking current consumers of the builder? | 16:28 |
clarkb | I don't know, I think that would be the next thing to figure out if indeed we're failing at the minimalness goal | 16:28 |
zbr | probably it would be easier to make the lookups relative to the executable location, this should be backwards compatible and allow others to copy files to different locations than /output | 16:30 |
clarkb | yes, I think that solves the backward compatibility issue. I do still think it may make it hard for people to do the right thing, but maybe we can address that with documentation in the short term | 16:30 |
clarkb | (then flip a default later or something) | 16:31 |
*** jpena is now known as jpena|off | 16:31 | |
zbr | i will update it. zuul is still building, that so slow to build. | 16:31 |
*** ysandeep is now known as ysandeep|away | 16:31 | |
clarkb | mostly my concern here is that we identify the problem to be solved and then articulate how the current thing helps. Rathe rthan just adding new features that may help a single user but leave everyone else building suboptimal images | 16:31 |
zbr | sure, avoiding the cost of options. | 16:32 |
zbr | it makes sense. | 16:32 |
zbr | btw, the newer buildkit option in docker does bring some huge improvements in build speed. | 16:33 |
clarkb | on the multiarch builds I think we're using the latest buildkit but maybe not on the single arch builds | 16:39 |
clarkb | (I think because buildx for multiarch implies buildkit | 16:39 |
*** hamalq has joined #openstack-infra | 16:53 | |
*** ykarel|away has joined #openstack-infra | 16:56 | |
*** lpetrut has quit IRC | 16:57 | |
*** Gyuseok_Jung has quit IRC | 17:00 | |
*** derekh has quit IRC | 17:00 | |
*** vishalmanchanda has joined #openstack-infra | 17:04 | |
*** ykarel|away has quit IRC | 17:05 | |
*** dklyle has quit IRC | 17:21 | |
*** david-lyle has joined #openstack-infra | 17:21 | |
zbr | ii checked, not big diff. if i have time i will rework it with relative dir. | 17:22 |
*** andrewbonney has quit IRC | 17:31 | |
*** gfidente is now known as gfidente|afk | 17:34 | |
*** dtantsur is now known as dtantsur|afk | 17:35 | |
*** Lucas_Gray has quit IRC | 17:41 | |
*** __ministry has joined #openstack-infra | 17:43 | |
*** __ministry has quit IRC | 17:43 | |
*** hamalq has quit IRC | 17:53 | |
*** hamalq has joined #openstack-infra | 17:53 | |
*** Jeffrey4l has quit IRC | 18:08 | |
AJaeger | config-core, please review https://review.opendev.org/745877 https://review.opendev.org/749688 https://review.opendev.org/716298 https://review.opendev.org/#/c/742732/ https://review.opendev.org/#/c/751130/ | 18:30 |
AJaeger | fungi, could you review https://review.opendev.org/#/c/749777/ - revert setuptool pin, please? | 18:30 |
fungi | approved. fingers crossed 50.3.0 hasn't reintroduced the problem | 18:31 |
AJaeger | ;) | 18:31 |
*** eolivare has quit IRC | 18:32 | |
clarkb | I'm pretty sure I rechecekd the system-config change on 50.3.0 and it was fine (it has the same issues as our image venvs) | 18:34 |
*** mgoddard has quit IRC | 18:34 | |
openstackgerrit | Merged openstack/project-config master: Revert "Pin setuptools<50 in our image venvs" https://review.opendev.org/749777 | 18:37 |
*** mgoddard has joined #openstack-infra | 18:41 | |
*** Jeffrey4l has joined #openstack-infra | 18:48 | |
*** zxiiro has quit IRC | 18:54 | |
*** Jeffrey4l has quit IRC | 18:55 | |
*** jtomasek has joined #openstack-infra | 18:55 | |
*** Jeffrey4l has joined #openstack-infra | 18:57 | |
*** jtomasek has quit IRC | 19:08 | |
*** tbachman has joined #openstack-infra | 19:12 | |
*** david-lyle is now known as dklyle | 19:14 | |
*** zzzeek has quit IRC | 19:14 | |
*** zzzeek has joined #openstack-infra | 19:16 | |
*** zzzeek has quit IRC | 19:20 | |
*** zzzeek has joined #openstack-infra | 19:23 | |
*** zzzeek has quit IRC | 19:28 | |
*** zzzeek has joined #openstack-infra | 19:29 | |
*** hamalq has quit IRC | 19:32 | |
*** hamalq has joined #openstack-infra | 19:33 | |
*** zzzeek has quit IRC | 19:33 | |
*** zzzeek has joined #openstack-infra | 19:35 | |
*** Topner has quit IRC | 19:39 | |
*** vishalmanchanda has quit IRC | 19:44 | |
*** slaweq_ has joined #openstack-infra | 19:53 | |
*** zzzeek has quit IRC | 19:54 | |
*** zzzeek has joined #openstack-infra | 19:57 | |
tbachman | was py27 taken out of the instances used for gate jobs recently? Like in the last week? | 20:02 |
clarkb | tbachman: I think that may have been part of gmann's focal work since focal doesn't have a python27 by default? | 20:02 |
clarkb | tbachman: but ya the switch to ubuntu focal for jobs removes python2.7. I think it may have an optionally installable 2.7 or you can keep running jobs on bionic that need 2.7 | 20:02 |
tbachman | k - I seem to recall being able to specify the instance OS type - will try that then | 20:03 |
tbachman | thanks clarkb! | 20:03 |
*** Topner has joined #openstack-infra | 20:29 | |
*** Topner has quit IRC | 20:34 | |
*** matt_kosut has quit IRC | 20:38 | |
*** slaweq__ has joined #openstack-infra | 20:55 | |
*** slaweq has quit IRC | 20:55 | |
*** Topner has joined #openstack-infra | 21:06 | |
*** slaweq__ has quit IRC | 21:08 | |
*** Topner has quit IRC | 21:11 | |
*** rfolco|ruck has quit IRC | 21:12 | |
*** kaisers2 has joined #openstack-infra | 21:37 | |
*** kaisers1 has quit IRC | 21:37 | |
*** slaweq_ has quit IRC | 21:38 | |
*** slaweq_ has joined #openstack-infra | 21:40 | |
*** zxiiro has joined #openstack-infra | 21:51 | |
*** slaweq_ has quit IRC | 21:53 | |
*** slaweq has joined #openstack-infra | 21:54 | |
*** slaweq has quit IRC | 22:05 | |
*** slaweq has joined #openstack-infra | 22:09 | |
*** slaweq has quit IRC | 22:21 | |
*** smarcet has quit IRC | 22:30 | |
*** tosky has quit IRC | 22:33 | |
*** tkajinam has joined #openstack-infra | 22:52 | |
*** Topner has joined #openstack-infra | 23:07 | |
*** Topner has quit IRC | 23:12 | |
*** portdirect has quit IRC | 23:13 | |
*** portdirect has joined #openstack-infra | 23:13 | |
*** artom has quit IRC | 23:46 | |
*** Goneri has quit IRC | 23:53 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!