| *** gyee has quit IRC | 00:01 | |
| fungi | i don't expect ensure-pip to influence the virtualenv provided by ensure-virtualenv at all, unless we use the pip provided by ensure-pip in ensure-virtualenv to install virtualenv from pypi | 00:02 |
|---|---|---|
| ianw | fungi : is that true on ubuntu using packages though, where the two may not be abstracted? | 00:06 |
| ianw | (virtualenv abstracted from the system pip) | 00:06 |
| ianw | because i guess, either by design or bug, they're not | 00:06 |
| fungi | can you elaborate? | 00:07 |
| ianw | well what we're seeing is that the packaged virtualenv on xenial can not create a virtualenv with the packaged pip on xenial in the opendev.org mirror environment right? | 00:08 |
| fungi | the ubuntu packaging provides basically two copies of pip: one executable/importable normally for use from the system context, and one bundled as a wheel for virtualenv and venv to bootstrap new environments with | 00:08 |
| *** artom has quit IRC | 00:08 | |
| fungi | it provides the former from the python-pip package and the latter from the python-pip-whl package | 00:08 |
| ianw | sure, but both of them are built from the same source package | 00:09 |
| fungi | correct | 00:09 |
| fungi | but they can be installed independently | 00:09 |
| fungi | the python-virtualenv package uses and depends on the python-pip-whl package, not the python-pip package | 00:09 |
| ianw | ok, sure, but ... i don't think that really helps does it. you're saying install python-pip-whl v9 with "ensure-virtualenv" maybe? | 00:10 |
| clarkb | ya the confusion here is fungi is saying the wheels are not a system package or it was implied earlier anyway | 00:10 |
| fungi | so our ensure-pip role installs python-pip, our ensure-virtualenv role installs (by way of dependency) the python-pip-whl package | 00:10 |
| clarkb | reality is they are both system packages with the same underlying problem | 00:10 |
| fungi | upgrading the pip provided by the ensure-pip role doesn't upgrade the pip wheel being used by virtualenv | 00:10 |
| clarkb | what ianw and I meant by system pip is "whatever it is that virtualenv uses to isntall pkg_resources comes from the distro" | 00:11 |
| ianw | fungi: ahh, but i think it does. it pulls in the python-pip-whl package | 00:11 |
| * fungi double-checks | 00:11 | |
| fungi | oh, yep | 00:11 |
| fungi | https://packages.ubuntu.com/xenial/python-pip says it also depends on python-pip-whl | 00:12 |
| fungi | with a versioned dependency | 00:12 |
| fungi | i expect that's to make ensure-pip happy | 00:12 |
| fungi | since https://packages.ubuntu.com/xenial/all/python-pip/filelist says that python-pip directly provides the pip package modules | 00:12 |
| fungi | so yes, you'll indirectly get a newer python-pip-whl when upgrading python-pip, due to versioned dependencies in python-pip for python-pip-whl | 00:13 |
| ianw | ok, i think we're in furious agreement :) | 00:14 |
| fungi | though installing python-pip just to upgrade the version of python-pip-whl for python-virtualenv seems unnecessary, you could just upgrade python-pip-whl and that wouldn't leave you with a pip executable in the path | 00:15 |
| fungi | and also wouldn't conflict with a separate install of pip from pypi or something | 00:15 |
| fungi | python-virtualenv also has a versioned dep on python-pip-whl, but it's >= so allows you to upgrade without being lock-step | 00:16 |
| ianw | if xenial were current i'd probably agree on some sort of workaround like that; but i feel like the most likely case of things right now is that having the pip binary as well probably isn't that bad | 00:16 |
| clarkb | ya having working pip is good too | 00:18 |
| clarkb | this same issue exists in both places | 00:19 |
| clarkb | and we want it working in both places | 00:19 |
| fungi | yeah, if it's only on xenial then i find it hard to care too much about correctness at this stage | 00:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:23 |
| ianw | fungi / clarkb: ^ i think that is it, in summary | 00:23 |
| *** stevebaker has joined #openstack-infra | 00:31 | |
| *** armax has quit IRC | 00:33 | |
| fungi | it's got a lot of build failures | 00:33 |
| *** yamamoto has quit IRC | 00:35 | |
| *** yamamoto has joined #openstack-infra | 00:35 | |
| ianw | - typo i think | 00:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:38 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 00:38 |
| ianw | oh, centos-7 is unhappy | 00:41 |
| ianw | i think we need to enable epel around the install | 00:42 |
| *** tetsuro has joined #openstack-infra | 00:42 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 00:53 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 00:53 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: install from EPEL for Python 2 https://review.opendev.org/736070 | 00:53 |
| ianw | clarkb/fungi: ^ sigh, i think that needs to stack under it. i think testing missed the python2 path on centos7 :/ | 00:53 |
| fungi | aha | 00:54 |
| *** Goneri has joined #openstack-infra | 00:54 | |
| ianw | don't we love dealing with centos7 and xenial issues :) | 00:54 |
| fungi | i can think of things i love slightly more | 00:55 |
| *** matt_kosut has joined #openstack-infra | 01:32 | |
| *** matt_kosut has quit IRC | 01:37 | |
| *** stevebaker has quit IRC | 01:38 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Remove the -plain job variants https://review.opendev.org/735774 | 01:48 |
| *** Goneri has quit IRC | 01:48 | |
| *** armax has joined #openstack-infra | 01:52 | |
| *** rosmaita has joined #openstack-infra | 02:01 | |
| *** ricolin has joined #openstack-infra | 02:01 | |
| *** Lucas_Gray has quit IRC | 02:12 | |
| *** lbragstad has quit IRC | 02:31 | |
| *** apetrich has quit IRC | 02:42 | |
| *** psachin has joined #openstack-infra | 02:48 | |
| *** rcernin has quit IRC | 02:49 | |
| *** stevebaker has joined #openstack-infra | 02:51 | |
| *** rcernin has joined #openstack-infra | 02:54 | |
| *** rcernin has quit IRC | 03:08 | |
| *** eharney has quit IRC | 03:30 | |
| *** psachin has quit IRC | 03:31 | |
| *** psachin has joined #openstack-infra | 03:32 | |
| *** armax has quit IRC | 03:40 | |
| *** rcernin has joined #openstack-infra | 03:45 | |
| *** rcernin has quit IRC | 03:45 | |
| *** rcernin has joined #openstack-infra | 03:50 | |
| *** eharney has joined #openstack-infra | 03:52 | |
| *** ysandeep|away is now known as ysandeep | 03:55 | |
| *** ykarel|away is now known as ykarel | 04:08 | |
| *** yamamoto has quit IRC | 04:19 | |
| *** markvoelker has joined #openstack-infra | 04:27 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** markvoelker has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-infra | 04:33 | |
| *** yamamoto has joined #openstack-infra | 04:35 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Remove the -plain job variants https://review.opendev.org/735774 | 04:48 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-pip: install from EPEL for Python 2 https://review.opendev.org/736070 | 04:53 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-virtualenv: call ensure-pip for Xenial https://review.opendev.org/736067 | 04:53 |
| *** diablo_rojo has quit IRC | 04:53 | |
| *** factor__ has joined #openstack-infra | 05:19 | |
| *** icarusfactor has quit IRC | 05:22 | |
| *** threestrands has joined #openstack-infra | 05:27 | |
| *** threestrands has quit IRC | 05:33 | |
| *** udesale has joined #openstack-infra | 05:33 | |
| *** matt_kosut has joined #openstack-infra | 05:34 | |
| *** matt_kosut has quit IRC | 05:38 | |
| *** lmiccini has joined #openstack-infra | 05:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack/openstack-zuul-jobs master: Ensure pip and tox are installed for announce-release https://review.opendev.org/736043 | 05:52 |
| AJaeger | ianw: I think that's better, can you confirm, please? ^ | 05:54 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Ensure pip is installed for propose-update-constraints https://review.opendev.org/736032 | 05:56 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Ensure pip is installed for propose-update-constraints https://review.opendev.org/736032 | 06:03 |
| *** matt_kosut has joined #openstack-infra | 06:05 | |
| ianw | AJaeger: thanks, yep 736043 seems to call out to a shell script so we don't have to keep digging and switch install to venv etc, leave it be | 06:06 |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Ensure pip and tox are installed for announce-release https://review.opendev.org/736043 | 06:18 |
| openstackgerrit | Merged openstack/project-config master: Turn -plain nodes down to min-ready 0 https://review.opendev.org/735777 | 06:32 |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-pip: remove Xenial venv workaround https://review.opendev.org/736068 | 06:37 |
| openstackgerrit | Merged openstack/project-config master: Replace build-sphinx-docs jobs https://review.opendev.org/736016 | 06:39 |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Update echo for pdf-docs to match recent change https://review.opendev.org/735819 | 06:41 |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Only run build-openstack-sphinx-docs on stable branches https://review.opendev.org/735937 | 06:41 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add vexxhost/ansible-role-frrouting https://review.opendev.org/736115 | 06:55 |
| *** rpittau|afk is now known as rpittau | 06:56 | |
| *** slaweq has joined #openstack-infra | 07:00 | |
| *** jcapitao has joined #openstack-infra | 07:01 | |
| *** amoralej|off is now known as amoralej | 07:07 | |
| *** apetrich has joined #openstack-infra | 07:11 | |
| *** ccamacho has joined #openstack-infra | 07:12 | |
| *** psachin has quit IRC | 07:15 | |
| *** psachin has joined #openstack-infra | 07:17 | |
| *** pkopec has joined #openstack-infra | 07:23 | |
| *** dklyle has quit IRC | 07:25 | |
| *** tosky has joined #openstack-infra | 07:36 | |
| *** nightmare_unreal has joined #openstack-infra | 07:41 | |
| *** alejandrosg has quit IRC | 07:45 | |
| *** alejandrosg has joined #openstack-infra | 07:46 | |
| *** rcernin has quit IRC | 07:47 | |
| openstackgerrit | Merged openstack/project-config master: Ensure pip is installed for propose-update-constraints https://review.opendev.org/736032 | 07:53 |
| *** jpena|off is now known as jpena | 07:55 | |
| *** ralonsoh has joined #openstack-infra | 07:57 | |
| *** pkopec has quit IRC | 07:58 | |
| *** pkopec has joined #openstack-infra | 08:02 | |
| *** ykarel is now known as ykarel|lunch | 08:13 | |
| *** ramishra has quit IRC | 08:17 | |
| *** gfidente|afk is now known as gfidente | 08:18 | |
| *** Lucas_Gray has joined #openstack-infra | 08:24 | |
| *** rakhmerov has joined #openstack-infra | 08:35 | |
| *** Lucas_Gray has quit IRC | 08:35 | |
| *** Lucas_Gray has joined #openstack-infra | 08:36 | |
| rakhmerov | AJaeger: hi, you recently landed a patch in Mistral after the "uwsgi" issue was somehow fixed in devstach. You said you were waiting for https://review.opendev.org/#/c/735494/ but I see it's still not merged | 08:40 |
| rakhmerov | now we're trying to land something into stable/queens (bindep.txt) and facing the same issue | 08:40 |
| rakhmerov | https://review.opendev.org/#/c/736129/ | 08:41 |
| rakhmerov | can you give any hints on how to fix it in queens? | 08:41 |
| AJaeger | rakhmerov: https://review.opendev.org/735523 fixed it - best discuss further in #openstack-qa | 08:42 |
| AJaeger | rakhmerov: AFAIK stable branches are still in progress | 08:43 |
| rakhmerov | ok | 08:43 |
| rakhmerov | thanks | 08:43 |
| *** priteau has joined #openstack-infra | 08:50 | |
| *** ociuhandu has quit IRC | 08:51 | |
| *** tkajinam has quit IRC | 08:57 | |
| *** ykarel|lunch is now known as ykarel | 09:03 | |
| *** apetrich has quit IRC | 09:17 | |
| *** apetrich has joined #openstack-infra | 09:26 | |
| *** ysandeep is now known as ysandeep|afk | 09:26 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 09:30 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 09:35 |
| *** ramishra has joined #openstack-infra | 09:37 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add os_adjutant zuul project https://review.opendev.org/736145 | 09:40 |
| *** gnuoy has quit IRC | 09:54 | |
| *** ysandeep|afk is now known as ysandeep | 09:54 | |
| *** yamamoto has quit IRC | 10:01 | |
| *** yamamoto has joined #openstack-infra | 10:02 | |
| *** yamamoto has quit IRC | 10:02 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 10:05 |
| *** jcapitao is now known as jcapitao_lunch | 10:15 | |
| *** derekh has joined #openstack-infra | 10:17 | |
| *** rpittau is now known as rpittau|bbl | 10:18 | |
| noonedeadpunk | AJaeger: can you help us a bit? We've faced with zuul error here https://review.opendev.org/723597 but really don't see what's wrong with patch... Can you take a brief look at it? | 10:24 |
| noonedeadpunk | disregard | 10:25 |
| noonedeadpunk | I found it | 10:25 |
| noonedeadpunk | by the way - there should be a hold for https://review.opendev.org/#/c/689629/ ? | 10:28 |
| noonedeadpunk | but I guess we have bunch of not ready roles for focal (in terms of non-core ones) | 10:31 |
| *** psachin has quit IRC | 10:31 | |
| *** yamamoto has joined #openstack-infra | 10:38 | |
| *** yamamoto has quit IRC | 10:39 | |
| *** yamamoto has joined #openstack-infra | 10:40 | |
| frickler | noonedeadpunk: yes, there's a held node, if you tell me your SSH key I can give you access | 10:41 |
| noonedeadpunk | frickler: sure https://launchpad.net/~noonedeadpunk/+sshkeys | 10:42 |
| noonedeadpunk | (regarding focal - message was posted int wrong channel, sorry) | 10:43 |
| frickler | noonedeadpunk: root@104.239.168.126 | 10:45 |
| noonedeadpunk | thanks, it works | 10:45 |
| frickler | iurygregory: infra-root: another ansible 2.9 failure: https://zuul.opendev.org/t/openstack/build/6195f1eada7c413495d515800aff1b78 | 10:47 |
| iurygregory | frickler, this one we have a fix | 10:48 |
| iurygregory | https://review.opendev.org/#/c/736124/ | 10:48 |
| *** Lucas_Gray has quit IRC | 10:49 | |
| frickler | iurygregory: ah, great, thx | 10:49 |
| iurygregory | np we are also backporting already to unblock stable/ussuri in ironic | 10:49 |
| *** yamamoto has quit IRC | 10:50 | |
| *** yamamoto has joined #openstack-infra | 11:00 | |
| *** derekh has quit IRC | 11:00 | |
| *** derekh has joined #openstack-infra | 11:00 | |
| *** psachin has joined #openstack-infra | 11:03 | |
| *** ociuhandu has joined #openstack-infra | 11:12 | |
| *** artom has joined #openstack-infra | 11:20 | |
| *** jpena is now known as jpena|lunch | 11:30 | |
| *** sshnaidm_ has joined #openstack-infra | 11:32 | |
| *** sshnaidm has quit IRC | 11:33 | |
| *** sshnaidm_ is now known as sshnaidm | 11:35 | |
| *** rfolco|rover has joined #openstack-infra | 11:36 | |
| *** tetsuro_ has joined #openstack-infra | 11:42 | |
| *** tetsuro has quit IRC | 11:46 | |
| *** tetsuro_ has quit IRC | 11:51 | |
| *** tosky has quit IRC | 11:56 | |
| *** tetsuro has joined #openstack-infra | 11:57 | |
| openstackgerrit | Slawek Kaplonski proposed openstack/project-config master: [Grafana] Update label of neutron-functional-with-uwsgi job https://review.opendev.org/736168 | 11:58 |
| *** psachin has quit IRC | 11:58 | |
| *** tosky has joined #openstack-infra | 11:59 | |
| *** psachin has joined #openstack-infra | 11:59 | |
| *** Lucas_Gray has joined #openstack-infra | 12:02 | |
| dtantsur | hey folks, the test-release-openstack job seems broken: https://zuul.opendev.org/t/openstack/build/d9ebbf199220436a9076600d6f9ce874/log/job-output.txt#445 | 12:02 |
| dtantsur | iurygregory: I think stable/train needs this patch too | 12:03 |
| iurygregory | dtantsur, yeah | 12:03 |
| dtantsur | "msg": "Missing sudo password" | 12:04 |
| dtantsur | this is what breaks test-release-openstack, apparently? | 12:04 |
| *** rpittau|bbl is now known as rpittau | 12:06 | |
| noonedeadpunk | ok, so we see pretty weird behaviour in centos ci image... which partially caused with this.. http://paste.openstack.org/show/794879/ | 12:08 |
| noonedeadpunk | And we don't see such behaviour in real centos images... | 12:08 |
| noonedeadpunk | maybe anyone can give some hint where to look for the thing? | 12:08 |
| *** amoralej is now known as amoralej|lunch | 12:09 | |
| AJaeger | dtantsur: see my comment on https://review.opendev.org/#/c/736001 | 12:10 |
| AJaeger | dtantsur: thanks for reporting | 12:10 |
| *** jcapitao_lunch is now known as jcapitao | 12:10 | |
| dtantsur | ah, cool, thanks for the link | 12:10 |
| AJaeger | mordred: that might be easier to test than the post job ^ | 12:11 |
| openstackgerrit | Merged zuul/zuul-jobs master: Record artifact checksums and signatures to stdout https://review.opendev.org/735929 | 12:15 |
| *** Lucas_Gray has quit IRC | 12:16 | |
| *** hashar has joined #openstack-infra | 12:18 | |
| noonedeadpunk | oh, /openstack is a separate mount point | 12:18 |
| *** Lucas_Gray has joined #openstack-infra | 12:19 | |
| noonedeadpunk | `/dev/xvde1 on /openstack type ext4 (rw,noatime,seclabel,nobarrier,data=writeback)` | 12:19 |
| noonedeadpunk | I believe it's not we who do such preparation | 12:20 |
| AJaeger | noonedeadpunk: I'm not aware that we do anything like this in our images | 12:21 |
| noonedeadpunk | http://paste.openstack.org/show/794885/ | 12:22 |
| noonedeadpunk | I'm pretty sure we can't "fake" that with ansible.... | 12:22 |
| noonedeadpunk | however... we can pickup drive and format it... (and use afterwards) | 12:23 |
| noonedeadpunk | But I thought that ususally vms have signle drive? | 12:23 |
| *** Wryhder has joined #openstack-infra | 12:23 | |
| *** Lucas_Gray has quit IRC | 12:24 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Revert "Make sure wheel is installed for python releases" https://review.opendev.org/736183 | 12:25 |
| elod | hi, it seems oslo.messaging uses tox before running ensure-tox (openstack-tox-py27 @ rocky, queens): https://zuul.opendev.org/t/openstack/build/cfcd1bebf5614f2db96f221b5369ab4c/log/job-output.txt#408-411 | 12:25 |
| elod | any suggestion how to solve it? | 12:26 |
| noonedeadpunk | AJaeger: yeah, I've found that it's not always the cse, thanks | 12:27 |
| *** Wryhder has quit IRC | 12:28 | |
| *** Lucas_Gray has joined #openstack-infra | 12:32 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Avoid become on build-python-release https://review.opendev.org/736185 | 12:32 |
| *** eolivare has joined #openstack-infra | 12:33 | |
| *** jpena|lunch is now known as jpena | 12:37 | |
| *** lbragstad has joined #openstack-infra | 12:39 | |
| *** Tengu has quit IRC | 12:39 | |
| *** Tengu has joined #openstack-infra | 12:40 | |
| fungi | noonedeadpunk: per the disk bullet at https://docs.opendev.org/opendev/infra-manual/latest/testing.html "There is at least 80GB of disk available. This disk may not all be exposed in a single filesystem partition and so not all mounted at /. Any additional disk can be partitioned, formatted and mounted by the root user..." | 12:43 |
| noonedeadpunk | Yeah, I guess we didn't noticed that previously as really have preparation step, which just work in weird way for centos8 | 12:44 |
| noonedeadpunk | (ie it's mounted with wrong options or smth) | 12:44 |
| fungi | you're mostly gong to see that on rackspace nodes, because there we get something like 20gb rootfs and 100gb ephemeral disk | 12:45 |
| fungi | maybe ovh too, i really can't recall | 12:45 |
| *** slaweq_ has joined #openstack-infra | 12:48 | |
| *** slaweq has quit IRC | 12:48 | |
| *** mtreinish has joined #openstack-infra | 12:50 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 12:51 |
| fungi | but we don't partition/format the additional volume at boot, we leave it up to jobs to do that, for example: https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/configure-swap/tasks/ephemeral.yaml | 12:52 |
| fungi | noonedeadpunk: ^ | 12:53 |
| noonedeadpunk | fungi: yeah, I already found that we also do partitioning, thanks) | 12:53 |
| fungi | okay, cool. let us know if you need help digging into anything there | 12:55 |
| *** slaweq_ is now known as slaweq | 13:03 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:06 |
| *** amoralej|lunch is now known as amoralej | 13:07 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Revert "Make sure wheel is installed for python releases" https://review.opendev.org/736183 | 13:10 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:12 |
| *** tetsuro has quit IRC | 13:15 | |
| *** udesale_ has joined #openstack-infra | 13:26 | |
| *** udesale has quit IRC | 13:28 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:35 |
| *** rpittau is now known as rpittau|brb | 13:39 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:40 |
| *** priteau has quit IRC | 13:40 | |
| *** ricolin has quit IRC | 13:40 | |
| *** rlandy is now known as rlandy|mtg | 13:41 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:44 |
| *** kberger_ has joined #openstack-infra | 13:45 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 13:45 |
| *** kberger_ has quit IRC | 13:46 | |
| *** kberger_ has joined #openstack-infra | 13:46 | |
| *** KeithMnemonic has quit IRC | 13:49 | |
| *** rpittau|brb is now known as rpittau | 13:53 | |
| *** lpetrut has joined #openstack-infra | 13:56 | |
| *** priteau has joined #openstack-infra | 13:56 | |
| *** yamamoto has quit IRC | 14:06 | |
| *** ysandeep is now known as ysandeep|brb | 14:07 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Fix missing repos for zuul tenant https://review.opendev.org/736226 | 14:08 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 14:09 |
| AJaeger | mnaser: as PTL, please also review https://review.opendev.org/736142 | 14:12 |
| noonedeadpunk | AJaeger: oh, and how to import existing stuff to the new repo? with upstream? | 14:12 |
| openstackgerrit | Ghanshyam Mann proposed openstack/openstack-zuul-jobs master: Revert "Work around uwsgi breakage" https://review.opendev.org/736230 | 14:12 |
| mnaser | noonedeadpunk: yeah you have to upload it somewhere like github and then we import it into gerrit | 14:12 |
| noonedeadpunk | as I have content for https://review.opendev.org/#/c/736142/ | 14:12 |
| mnaser | noonedeadpunk: so throw it on github and update upstream (just make sure no zuul config anywhere in it) | 14:13 |
| noonedeadpunk | https://github.com/elastx/openstack-ansible-os_adjutant | 14:13 |
| *** gshippey_ has joined #openstack-infra | 14:13 | |
| mnaser | noonedeadpunk: yeah in that case it's best to update the upstream :) | 14:13 |
| AJaeger | noonedeadpunk: all documented in the Creator's guide, see https://docs.opendev.org/opendev/infra-manual/latest/creators.html | 14:14 |
| mnaser | https://docs.opendev.org/opendev/infra-manual/latest/creators.html#add-the-project-to-the-master-projects-list #3 | 14:14 |
| noonedeadpunk | mnaser: I'm just not sure - wouldn't it mean that it would sync back to upstream all new patches? | 14:15 |
| mnaser | noonedeadpunk: nope -- just a one time import | 14:15 |
| mnaser | so essentially forking it into opendev | 14:15 |
| noonedeadpunk | ah, ok. that's why I didn't add at first place | 14:15 |
| mnaser | noonedeadpunk: the only concern is that repo has a zuul.d folder. i think it _may_ contain valid config but we may disallow that, we may ask to remove that first. AJaeger might know more :X | 14:15 |
| mnaser | i remember we have a "dont import if there's zuul config" rule | 14:16 |
| noonedeadpunk | I'm pretty sure zuul.d is not alid there... | 14:16 |
| noonedeadpunk | *valid | 14:16 |
| mnaser | yeah maybe we'll have to do a PR then | 14:17 |
| noonedeadpunk | let me set -W then... | 14:17 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 14:18 |
| noonedeadpunk | btw, will it clone branches as well? | 14:19 |
| noonedeadpunk | (I guess yes) | 14:19 |
| AJaeger | mnaser: yes, we have a check that will -1 if t here's a zuul.d folder. noonedeadpunk, remove it and then recheck. Also, it clones *all* branches. If you don't want them, delete - or create a fork without them and import that one... | 14:22 |
| openstackgerrit | Merged openstack/project-config master: Add vexxhost/ansible-role-frrouting https://review.opendev.org/736115 | 14:23 |
| mnaser | AJaeger: i think that might be good for us to document in there | 14:23 |
| mnaser | it seems to just be tribal info not in the infra-manual | 14:23 |
| noonedeadpunk | Maybe it's easier just to push content afterwards? | 14:23 |
| mnaser | noonedeadpunk: if we push content afterwards, then it will squash and lose all history | 14:24 |
| noonedeadpunk | like trough gerrit | 14:24 |
| AJaeger | mnaser: want to sent a patch for opendev/infra-manual? | 14:24 |
| noonedeadpunk | folks don't care about it | 14:24 |
| mnaser | you'd have to make a ton of gerrit reviews :p | 14:24 |
| mnaser | i can throw something up quickly | 14:24 |
| mnaser | if they dont care about history, you can squash locally and push it all in 1 patch after. it's your choice | 14:24 |
| noonedeadpunk | Like I had a chat with them and they don't mind if we just drop all history | 14:24 |
| openstackgerrit | Merged openstack/project-config master: Add noop-jobs for neutron-fwaas and neutron-fwaas-dashboard projects https://review.opendev.org/735807 | 14:25 |
| fungi | noonedeadpunk: mnaser: AJaeger: we import all branches *and* tags too | 14:29 |
| fungi | so if there are tags you don't want you should also delete those | 14:30 |
| fungi | and you don't have to use github, it's just an example. i scp a local git repo to a webserver and point at the url. anything which can be publicly cloned from will work fine for the import process | 14:30 |
| AJaeger | fungi: let's document in 736238 | 14:32 |
| *** yamamoto has joined #openstack-infra | 14:34 | |
| *** yamamoto has quit IRC | 14:34 | |
| *** yamamoto has joined #openstack-infra | 14:34 | |
| *** psachin has quit IRC | 14:34 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 14:36 |
| *** rlandy|mtg is now known as rlandy | 14:39 | |
| *** yamamoto has quit IRC | 14:41 | |
| *** dabukalam is now known as danny | 14:43 | |
| *** danny is now known as dabukalam | 14:43 | |
| *** dklyle has joined #openstack-infra | 14:44 | |
| clarkb | elod: did you manage to get that tracked down? | 14:49 |
| AJaeger | fungi, could you put the infra-manual change 736238 on your review queue, please? | 14:49 |
| clarkb | elod: that looks like a bug either in the job where test-setup runs before ensure-tox or if that is necessary, a bug in the test setup assuming tox is there | 14:49 |
| clarkb | elod: I don't know enough yet to say which thing should be fixed but my hunch is we should run test-setup after ensure-tox | 14:50 |
| fungi | AJaeger: sure | 14:51 |
| fungi | thanks for the heads up | 14:51 |
| AJaeger | elod, btw. that test-setup.sh is completely redundant and could be removed. | 14:52 |
| elod | clarkb: there's no bug in the job i think, but oslo-messaging have 'tox -e bindep' in their tools/test-setup.sh | 14:53 |
| elod | AJaeger: hmmm. is it unnecessary in stable/rocky, too? | 14:53 |
| AJaeger | elod: everywhere... | 14:53 |
| fungi | i guess it's there to remind devs if they're missing packages? | 14:53 |
| AJaeger | elod: https://review.opendev.org/736248 | 14:53 |
| AJaeger | fungi: it basically calls bindep... | 14:54 |
| elod | AJaeger: good to know | 14:54 |
| fungi | yeah, i mean, the alternative is to say "run `tox -e bindep`" in the readme | 14:55 |
| elod | AJaeger: that looks good then! Thanks! I was trying to figure out where to put ensure-tox or how to move bindep check, but then your fix is the simplest :) | 14:55 |
| AJaeger | elod: still, we could change the order ... | 14:55 |
| *** lmiccini has quit IRC | 14:56 | |
| clarkb | AJaeger: elod the ordering there could probably be changed | 14:56 |
| clarkb | since as much setup before test-setup as is safe is likely a good thing | 14:56 |
| clarkb | however tox inherits from unittests and that is what sets the order | 14:56 |
| clarkb | changing that would be difficult/weird | 14:56 |
| clarkb | so we may have to live with the ordering | 14:57 |
| AJaeger | mnaser: https://review.opendev.org/#/c/736142 is fine, I'll +2 once the PTL (you ;) has reviewed. | 14:57 |
| *** Goneri has joined #openstack-infra | 14:57 | |
| mnaser | AJaeger: +2 | 14:58 |
| clarkb | I'm beginning to wonder if we shouldn't put an etherpad together with some rough explanation of various classes of issue and related fixes | 14:58 |
| clarkb | though I've just sat down and not sure how prevalent problems continue to be | 14:58 |
| AJaeger | mnaser: +3 ;) | 14:58 |
| AJaeger | clarkb: if we create an etherpad, there won't be many - if not,... | 14:59 |
| clarkb | we have the ansible 2.9 issues, the pip change issues, and the large cleanup for grenade and devstack related to uwsgi and disabling jobs | 14:59 |
| clarkb | looks like the devstack and grenade cleanup is starting with https://review.opendev.org/#/c/736230/1 so we must be looking better there | 15:00 |
| fungi | and there's an outstanding fix needed to install wheel before building python packages, which is the last known remaining blocker to get openstack releases going again | 15:01 |
| fungi | in progress at https://review.opendev.org/735905 | 15:02 |
| clarkb | fungi: wasn't that change reverted? | 15:03 |
| clarkb | AJaeger: mnaser: why is it important that there is no zuul config in the imported project? simply because it is unlikely to work and we want to avoid having first commit be "cleanup zuul config" ? | 15:03 |
| fungi | clarkb: https://review.opendev.org/736185 sorry | 15:03 |
| mnaser | clarkb: if i remember right, a while back someone managed to import a bad zuul config that caused issues in zuul (because it wasn't gated) | 15:04 |
| *** ysandeep|brb is now known as ysandeep | 15:04 | |
| fungi | clarkb: and yes, the original was reverted because it didn't work, and worse broke the job for folks who did still have wheel preinstalled | 15:04 |
| mnaser | i think maybe times were different and zuul's ability to deal with broken configs wasn't so strong. i briefly remember it | 15:04 |
| fungi | mnaser: zuul can still get pretty confused if we install additional configuration which defines some of the same jobs and other objects whose names it assumes should be globally unique | 15:05 |
| clarkb | ya I guess my concern with the documentation is that its not strictly true. But for guidance to people importing projects it is probably the best advice anyway | 15:06 |
| fungi | for example if we import a new repository which includes jobs with the same names as ones relied on by lots of our projects, that could get disruptive easily i think | 15:06 |
| clarkb | better to build up zuul config with it being properly tested than import and hope it works | 15:06 |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Revert "Work around uwsgi breakage" https://review.opendev.org/736230 | 15:08 |
| fungi | we might be able to pre-test imports if we separated the creation and import steps | 15:08 |
| fungi | though that would probably still be really complicated to pull off | 15:09 |
| AJaeger | clarkb: we have a test in project-config that gives a -1 if zuul files get imported. We added this to protect us from really bad configs, we wanted them to come in via gerrit review | 15:09 |
| fungi | like start with an empty project, then have it propose a squashed change of each branch tip, and only import if they all passed their jobs or something | 15:10 |
| clarkb | AJaeger: cool. I did approve the change already fwiw. Just awnted to make sure i understood the guidance there | 15:10 |
| AJaeger | clarkb: thx | 15:11 |
| openstackgerrit | Merged openstack/project-config master: Add openstack-ansible-os_adjutant repo https://review.opendev.org/736142 | 15:12 |
| *** Lucas_Gray has quit IRC | 15:12 | |
| clarkb | noonedeadpunk: did you get your held node afterall? | 15:13 |
| clarkb | seems like there was discussion of rax disk partitioning, I assume that is related? | 15:13 |
| fungi | yes, possible he's done with the held node now | 15:17 |
| *** sshnaidm is now known as sshnaidm|bbl | 15:17 | |
| fungi | noonedeadpunk: do let us know once you finish studying that node so we can release it back into the wild | 15:17 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: WIP: Avoid become on build-python-release https://review.opendev.org/736185 | 15:19 |
| ttx | quick question regarding jeepyb -- how often is the manage_projects command run? | 15:20 |
| clarkb | ttx: when we land changes to projects.yaml iirc | 15:21 |
| clarkb | ttx: I'm not sure if it is part of periodic updates still /me checks | 15:21 |
| ttx | and then it updates all repos? Or just the changed one? | 15:21 |
| noonedeadpunk | fungi: I will finish today, sorry, got some urgent thing to finish... | 15:22 |
| ttx | clarkb: Trying to see how much I need to create github repositories for the mirroring work | 15:22 |
| clarkb | ttx: it will apply to all repos. Also just confirmed it doesn't seem to be periodic but runs when a few other things change like our inventory and gitea roles and so on | 15:22 |
| AJaeger | so, does ttx need to do manual work for https://review.opendev.org/#/c/736145/ ? | 15:22 |
| AJaeger | Or could we just merge that? | 15:22 |
| clarkb | ttx: but the major change that should drive it is updates to the projects.yaml input to the command | 15:23 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add os_adjutant zuul project https://review.opendev.org/736145 | 15:23 |
| fungi | noonedeadpunk: no worries, take all the time you need | 15:23 |
| ttx | ok | 15:23 |
| noonedeadpunk | clarkb: yeah, I got it, thanks :) | 15:23 |
| clarkb | AJaeger: ttx that change is a zuul config update? shouldn't be related to manage projects at all? | 15:23 |
| AJaeger | clarkb: it adds the github mirroring template for a repo that was just created a few minutes ago | 15:24 |
| clarkb | https://review.opendev.org/#/c/736142/5/gerrit/projects.yaml right that is the file that will cause manage projects to run | 15:24 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks https://review.opendev.org/731606 | 15:24 |
| clarkb | if you look at zuul status now you can see manage projects has completed successfully | 15:24 |
| clarkb | for that change | 15:24 |
| AJaeger | clarkb: so, I wonder whether ttx as github admin needs to do anything or whether we can merge 736145 now and it will mirror fine | 15:24 |
| * ttx checks if the corresponding git repo was created on github | 15:25 | |
| fungi | the mirror job will just report failures until the repo is created | 15:25 |
| fungi | but should be okay to add it | 15:25 |
| ttx | AJaeger: yeah, that's what I was trying to check. I assumed I had to create things on the github side | 15:26 |
| ttx | but now I see no reason why manage_projects would not do it | 15:27 |
| AJaeger | ttx, I wanted to double check whether I needed to wait on you doing it ;) | 15:27 |
| ttx | (just realized it was still running as it overwrote my customized description) | 15:27 |
| AJaeger | ttx, I'm not sure whether we have documented adding the openstack template and who's job that is - we might have merged some repos without adding it. Could you double check, please? | 15:29 |
| ttx | on it | 15:30 |
| AJaeger | thanks! | 15:30 |
| ttx | clarkb: so the manage_projects ran for that change, apparently: https://zuul.openstack.org/build/e74357ee0afc46ebb2df854cb0a80a67/log/manage-projects.yaml.log#100 | 15:32 |
| fungi | the main reason i could see for not doing it in manage-projects is we're trying to move more of that to zuul jobs, and it would be possible for a zuul job provided with the correct secret to call the gh api to create a repository | 15:33 |
| ttx | but then I can't find the corresponding git repo on github side | 15:33 |
| ttx | could be that it's invisible until pushed to, checking | 15:33 |
| ttx | hhm, no | 15:34 |
| fungi | oh, i thought we had stopped creating github repos with manage-projects and you were talking about reenabling it | 15:34 |
| ttx | which is consistent with recent repos I had to create. | 15:34 |
| ttx | fungi: but it does overwrite descriptions I set, so it's certainly running | 15:34 |
| ttx | maybe it just silently fails to create new repos | 15:35 |
| fungi | ahh, okay, so i guess we haven't yet managed to rip the github interactions out of our tools like we had discussed | 15:35 |
| clarkb | ya itmay be failing to createbut succeding on updates | 15:35 |
| clarkb | we can disable it entirely with a config flag iirc | 15:36 |
| fungi | right, i was thinking we had already disabled it, but i suppose not | 15:36 |
| ttx | fungi: also it still closes pull requests, which is good as it makes my corresponding work to do that less urgent :) | 15:36 |
| *** hashar has quit IRC | 15:38 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Update guideline docs for os specific tasks https://review.opendev.org/731606 | 15:40 |
| ttx | clarkb: yes, has-github | 15:41 |
| ttx | looking into that | 15:42 |
| ttx | meh that should have done it https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/gerrit/templates/projects.ini.j2 | 15:44 |
| fungi | we can presumably manually rerun m-p for that one repo with --verbose and see if it produces any obvious errors or doesn't log actions we think it should be taking | 15:48 |
| ttx | fungi: looking at the code it should definitely not run any of that repo creation, which is consistent with what we observe | 15:49 |
| ttx | so maybe I just dreamed that it overwrites my descriptions | 15:50 |
| *** udesale_ has quit IRC | 15:53 | |
| ttx | though it certainly still closes pull requests... and I'm not sure why it would | 15:53 |
| fungi | has-pull-requests might be evaluated separately? | 15:57 |
| ttx | the PR-closer evaluates use-github first and shortcircuits the rest | 15:58 |
| *** gyee has joined #openstack-infra | 15:58 | |
| ttx | https://opendev.org/opendev/jeepyb/src/branch/master/jeepyb/cmd/close_pull_requests.py#L107 | 15:59 |
| fungi | huh, there goes my theory then | 15:59 |
| ttx | yes and it was still closing PRs very recently. Maybe that job is still running off some other code | 15:59 |
| * ttx is down the rabbit hole | 16:00 | |
| ttx | not sure where close_pull_requests is actually run from | 16:01 |
| fungi | the gerrit container on review.o.o has jeepyb==0.0.1.dev469 # git sha ab498db | 16:02 |
| ttx | not much showing in http://codesearch.openstack.org/?q=close_pull&i=nope&files=&repos= | 16:02 |
| ttx | oh well, I'll continue that exploration tomorrow | 16:04 |
| fungi | the setup.cfg for jeepyb says close-pull-requests is an executable wrapper for the jeepyb.cmd.close_pull_requests:main entrypoint | 16:04 |
| ttx | fungi: any idea which job actually calls that? | 16:05 |
| fungi | a cronjob installed by puppet: https://opendev.org/opendev/puppet-github/src/branch/master/manifests/init.pp#L88-L97 | 16:06 |
| ttx | it still closed PRs two days ago | 16:06 |
| ttx | hah - vs _ | 16:06 |
| fungi | yeah, the console script uses - | 16:06 |
| fungi | but - is not viable in a module name | 16:07 |
| *** armax has joined #openstack-infra | 16:08 | |
| ttx | hmm so maybe this one does not see use-github=false. I'll dog from there tomorrow, thanks for the pointer | 16:09 |
| ttx | dig* | 16:09 |
| fungi | on review.o.o, `sudo crontab -l -u github` tells me that we do indeed still have this: | 16:10 |
| fungi | */5 * * * * sleep $((RANDOM\%60+90)) && /usr/local/bin/close-pull-requests | 16:10 |
| fungi | mordred: ^ i expect that's something we missed in the containerification of the gerrit server? | 16:10 |
| fungi | (either missed containerizing or missed cleaning up) | 16:11 |
| fungi | it's entirely possible that's working off a stale copy of project data now | 16:11 |
| *** dtantsur is now known as dtantsur|afk | 16:12 | |
| ttx | yeah that would explain it. The idea would be to have an openstack periodic zuul job that would create the missing repos and close pull requests... like once a day | 16:12 |
| ttx | but not driven by projects.yaml, openstack-specific | 16:13 |
| mordred | fungi: whoops | 16:15 |
| *** rpittau is now known as rpittau|afk | 16:17 | |
| fungi | i can easily comment that out and it won't get recreated since we no longer apply the puppet-github module (or indeed any puppet) on that server, but the question remains are there users relying on that and, if so, is that a reason to prioritize a replacement solution (such as the zuul job ttx mentions) | 16:19 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Make build-python-release use python3 https://review.opendev.org/736185 | 16:19 |
| clarkb | fungi: ttx mordred we get emails for github PRs and the rate is low | 16:23 |
| ttx | fungi: I think it's fine to remove now. I will quickly add PR-closing to my current github-update script and run it manually until I find time to turn it into a job | 16:23 |
| clarkb | but openstack likely should have some plan for dealing with those | 16:23 |
| fungi | clarkb: yes, and more than half are accidental git push from openstack contributors who happen to have forked from the gh mirror for $reasons and forgot which workflow they should use | 16:25 |
| fungi | we see maybe one or two legitimate pr attemps a month for openstack repos, if that | 16:25 |
| ttx | yeah, maybe we should turn those into first-contact entry points. | 16:27 |
| ttx | (the legitimate ones) | 16:27 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: POC: test build-python-release using python https://review.opendev.org/736293 | 16:29 |
| *** ykarel is now known as ykarel|away | 16:33 | |
| *** ysandeep is now known as ysandeep|away | 16:36 | |
| *** ociuhandu_ has joined #openstack-infra | 16:38 | |
| *** ociuhandu has quit IRC | 16:42 | |
| fungi | i agree. treat them similarly to the welcome new contributor reviews | 16:42 |
| *** ociuhandu_ has quit IRC | 16:43 | |
| fungi | also have a better story around accepting their patches and having an established contributor do a `gti am` manually if they'll add signed-off-by in the commit message | 16:43 |
| fungi | er, `git am` | 16:43 |
| *** ociuhandu has joined #openstack-infra | 16:47 | |
| *** ociuhandu has quit IRC | 16:51 | |
| *** amoralej is now known as amoralej|off | 16:55 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 16:56 |
| EmilienM | AJaeger, mnaser : thank you for all the help with paunch removal | 16:59 |
| mnaser | EmilienM: you're welcome. hope you've been well :) | 17:00 |
| EmilienM | mnaser: everything is good :-) | 17:00 |
| *** derekh has quit IRC | 17:01 | |
| *** lpetrut has quit IRC | 17:02 | |
| *** jpena is now known as jpena|off | 17:06 | |
| *** jcapitao has quit IRC | 17:11 | |
| *** kberger_ has quit IRC | 17:20 | |
| *** nightmare_unreal has quit IRC | 17:24 | |
| *** KeithMnemonic has joined #openstack-infra | 17:25 | |
| *** KeithMnemonic has quit IRC | 17:27 | |
| *** KeithMnemonic has joined #openstack-infra | 17:37 | |
| *** KeithMnemonic has quit IRC | 17:43 | |
| smcginnis | Hey kind infra folks - I did a test release to see if our issues are cleared up, and while things look better than they were, we did hit a failure on announce-release. | 17:58 |
| smcginnis | https://zuul.opendev.org/t/openstack/build/52cec40de6df4532822aa151f1ca2e4e | 17:58 |
| fungi | checking into that, thanks smcginnis | 17:58 |
| smcginnis | Looks like tox and/or virtualenv are missing, but that *should* have been fixed by https://review.opendev.org/#/c/736043/2/playbooks/release/pre.yaml | 17:58 |
| smcginnis | The log shows both prior to attempting to send the announcement. | 17:59 |
| fungi | yeah, seems like we need ensure-tox in the pre phase for announce-release | 18:00 |
| fungi | it looks like it tries to use tox to build a virtualenv, so the "Failed to build virtualenv" message is slightly misleading | 18:00 |
| smcginnis | fungi: Do you mean this pre phase: https://review.opendev.org/#/c/736043/2/playbooks/release/pre.yaml | 18:01 |
| smcginnis | Or is there a pre-pre phase? | 18:01 |
| fungi | oh, i wonder if it's just needing to use the variable which tells it where tox is installed | 18:02 |
| *** __ministry1 has joined #openstack-infra | 18:02 | |
| *** __ministry1 has quit IRC | 18:02 | |
| smcginnis | We may need to refactor some things then. | 18:03 |
| fungi | or symlink something into $PATH | 18:03 |
| smcginnis | The playbook calls the tools/announce.sh script in the releases repo. | 18:03 |
| smcginnis | Which internally calls tox. | 18:03 |
| fungi | clarkb: ^ how have we been handling that in other jobs? do you happen to know? | 18:03 |
| smcginnis | So yeah, I think we need it added to $PATH | 18:03 |
| fungi | i know we discussed symlinks as an option | 18:03 |
| smcginnis | Looking at the script, if it sees it is already in a $VIRTUAL_ENV it won't try to call it. | 18:04 |
| smcginnis | Ah, but it does that to have our release-notes cmd installed. | 18:06 |
| mordred | fungi: we set an ansible fact | 18:06 |
| mordred | so other ansible things can reference it | 18:06 |
| fungi | mordred: but if you have a shell script referencing it instead, what's our preference for solving that? | 18:06 |
| mordred | I think what we probably want to do is have the playbook that's calling teh release script add the variable to the path for the shell command | 18:06 |
| smcginnis | Kinda looks like that's what we need to do. | 18:06 |
| mordred | fungi: we do not have a solution for that - because _complicated_ interactions between non-root use and path setting and shells and ansible | 18:07 |
| fungi | right, i know we had discussed a number of possible options (appending to $PATH, symlinking into somewhere like /usr/local/bin) just couldn't remember if we decided one was better than another | 18:08 |
| mordred | fungi: yeah - unfortiunately we run in to 2 issues | 18:08 |
| mordred | one is that we run these installs as not-root, so we can't put in /usr/local/bin/ scripts | 18:08 |
| mordred | we _could_ write to ~/.profile or something - but it seems that's not getting read when ansible runs shell commands | 18:08 |
| openstackgerrit | Merged openstack/project-config master: Normalize projects.yaml https://review.opendev.org/735782 | 18:08 |
| mordred | so that didn't work out | 18:09 |
| fungi | oh, we're installing tox as a non-root user? | 18:09 |
| mordred | yeah | 18:09 |
| fungi | got it | 18:09 |
| mordred | we do as many of the installs as we can as non-root | 18:09 |
| mordred | so that the jobs also work for people running them in containres that dont' have root | 18:09 |
| mordred | just to make it extra complex | 18:09 |
| mordred | so what we do is set an ansible fact to cacheable and put in the virtualenv path to the executable | 18:09 |
| mordred | (iirc) | 18:09 |
| fungi | so we're creating the venvs in user-specific locations on the filesystem, not somewhere generally accessible to all users (though if the job only uses one user, i guess that's fine) | 18:09 |
| mordred | then since it's cacheable it gets picked up in later playbooks | 18:10 |
| mordred | fungi: that's right | 18:10 |
| fungi | default ~/.profile probably includes ~/bin in $PATH if it exists | 18:10 |
| mordred | I think we may have tried and determined we can't count on that | 18:10 |
| mordred | and/or I think ansible overwrites PATH in the ssh command | 18:10 |
| fungi | yeah, that tends to vary by distro | 18:10 |
| mordred | fungi: tox_executable is the ansible variable containing the path to the tox executable | 18:11 |
| fungi | the .profile addition of ~/bin varies by distro i mean | 18:11 |
| fungi | does adding tox_executable to $PATH work, or do we need to add basedir(tox_executable) instead? | 18:11 |
| mordred | basedir | 18:12 |
| mordred | it'll be like tox_venv/bin/tox | 18:12 |
| mordred | fungi: ensure_pip_virtualenv_command is a variable containing the virtualenv command in case that's helpful | 18:12 |
| fungi | that could get tricky, because there are likely to be additional executables in there we don't want to shadow system executables, so i guess we append it | 18:12 |
| fungi | rather than prepend | 18:13 |
| smcginnis | If there's an environment variable our release script can check, we could do that. | 18:13 |
| fungi | oh, sure, i mean there's a bunch of ways to tackle it if we alter the script | 18:14 |
| fungi | i was just trying to work out what a good solution might be without altering the script | 18:14 |
| smcginnis | It does seem like an issue that we could hit in other places, but if path of least resistance is to workaround it on our end, we can do that. | 18:14 |
| smcginnis | Or we add these back in the base images. ;) | 18:15 |
| mordred | smcginnis: :) | 18:15 |
| fungi | i guess the reason the script is calling tox is to get the project itself and its runtime deps installed into the venv where it's going to generate the announcement? | 18:16 |
| smcginnis | Yep. Looks like there are a couple of local cmds it needs to use to extract and send the announcement details. | 18:17 |
| mordred | fungi: you know ... | 18:19 |
| mordred | maybe what we should do is add a flag to ensure-tox like "tox_install_globally" or something | 18:20 |
| fungi | mordred: so for your suggestion, we'd add a path parameter to https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/playbooks/release/announce.yaml and use a j2 subtitution to pass tox_executable through some filter to get the dirname? | 18:20 |
| mordred | that people can set in contexts where they know they're running ensure-tox in a having-root crcumstance | 18:20 |
| fungi | er, i mean for your earlier suggestion | 18:20 |
| mordred | fungi: and then this job could just set the variable and have the tox in question symlinked to /usr/local/bin | 18:20 |
| mordred | fungi: re: earlier suggestion - yes | 18:21 |
| fungi | i guess path manipulation is a common parameter of the command module | 18:21 |
| mordred | corvus: ^^ scrollback here - what do you think about adding a variable like "tox_install_globally" - or even "ensure_global" that would cause ensure-tox (or ensure roles in general) to add a symlink into /usr/local/bin - not have that run by default, because of the rootless container issue | 18:22 |
| mordred | but for users who want their jobs to install stuff in places where scripts can use them without having to pass ansible variables to paths of shell comamnds | 18:22 |
| mordred | and who can assert that it's ok for the ensure roles to do that - maybe it's useful? | 18:23 |
| fungi | https://docs.ansible.com/ansible/latest/modules/command_module.html doesn't advertise path manipulation, or even environment manipulation, options | 18:23 |
| fungi | do we need shell instead? | 18:23 |
| mordred | fungi: it's not a command parameter | 18:23 |
| mordred | it's a top-level task param | 18:23 |
| mordred | one sec ... | 18:23 |
| fungi | aha, okay | 18:23 |
| fungi | yeah, i guess i shouldn't have assumed only things run by the command module care about envvars | 18:23 |
| fungi | and even the path envvar in particular | 18:24 |
| mordred | fungi: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/multi-node-bridge/tasks/common.yaml#L126 | 18:24 |
| corvus | mordred: oh wow. i ack your ping and will start reading scrollback. | 18:25 |
| fungi | thanks mordred | 18:25 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:28 |
| mordred | corvus: ^^ something like that handwave handwave | 18:28 |
| mordred | and then fungi could just add tox_ensure_global: true to this release job and the script should just work | 18:29 |
| corvus | mordred: i like it | 18:29 |
| *** hashar has joined #openstack-infra | 18:29 | |
| corvus | and reading all that scrollback helped | 18:29 |
| mordred | corvus: do you think we should try to mkae it a more global flag, like just ensure_global | 18:29 |
| mordred | or per-ensure role? | 18:29 |
| corvus | mordred: i'm still kind-of digging my "do both" idea from earlier, what do you think? | 18:30 |
| corvus | tox_ensure_global defaults to ensure_global defaults to false | 18:30 |
| corvus | though honestly... maybe that's overengineering | 18:30 |
| corvus | is a job going to have more than one user of ensure_global? | 18:30 |
| fungi | anybody know off the top of their heads what the ansible filter equivalent is for os.path.basedir()? | 18:31 |
| mordred | corvus: yeah - I like the do both idea ... but also maybe yeah, maybe it's too much for this | 18:32 |
| mordred | corvus: like, perhaps we should start with tox_ensure_global and see how many more we grow | 18:32 |
| corvus | fungi: what's basedir? do you mean basename? | 18:33 |
| mordred | or - eek, I keep talking myself around in circles | 18:33 |
| fungi | er, yep sorry | 18:33 |
| fungi | dirname | 18:33 |
| corvus | that's 'basename' https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html | 18:33 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:33 |
| corvus | fungi: dirname too | 18:33 |
| mordred | fungi: I think you can probably just depends-on that ^^ | 18:33 |
| *** eolivare has quit IRC | 18:33 | |
| mordred | (as soon as we decide what the variable name should be) | 18:33 |
| fungi | corvus: thanks. i would help if i didn't constantly get the function name for that wrong ;) | 18:34 |
| mordred | fungi: I get the name of that function wrong all the time | 18:34 |
| corvus | mordred: i think the test is whether a user would ever want different values for different roles | 18:34 |
| fungi | mordred: cool, i'll abandon the path manipulation attempt in that case | 18:34 |
| corvus | mordred: i can easily see that for the python version thing. i'm drawing a blank on a use-case for that for installing global symlinks. | 18:34 |
| mordred | corvus: same | 18:35 |
| mordred | like - if you're ok installing global symlinks - I imagine you'd want to do it for all the things | 18:35 |
| corvus | ya | 18:35 |
| mordred | corvus: so how about "ensure_global_symlinks" | 18:36 |
| mordred | or should we make it zuul_ensure_global_symlinks | 18:36 |
| corvus | ++ | 18:36 |
| corvus | oh um. | 18:36 |
| corvus | no zuul i think. | 18:36 |
| mordred | kk | 18:36 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:37 |
| openstackgerrit | Jeremy Stanley proposed openstack/openstack-zuul-jobs master: Symlink tox into the exec path for release jobs https://review.opendev.org/736334 | 18:40 |
| fungi | mordred: ^ | 18:40 |
| fungi | first user! | 18:40 |
| *** gfidente is now known as gfidente|afk | 18:40 | |
| fungi | smcginnis: ^ | 18:40 |
| mordred | fungi: \o/ | 18:44 |
| *** jamesmcarthur has joined #openstack-infra | 18:47 | |
| mordred | fungi: do we need to do a simlar thing with ensure-virtualenv? or is that just installing the package? | 18:47 |
| fungi | mordred: i don't think the script is directly calling virtualenv/venv | 18:50 |
| fungi | it just gives a somewhat misleading error message because it's using tox to create a virtualenv | 18:51 |
| fungi | so when the script can't find/run tox it says that it "failed to create the virtualenv" | 18:51 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:52 |
| mordred | fungi: awesome | 18:54 |
| mordred | fungi: you might need a recheck - I forgot a become: | 18:54 |
| *** jamesmcarthur has quit IRC | 18:54 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 18:55 |
| *** jamesmcarthur has joined #openstack-infra | 18:55 | |
| fungi | mordred: yep, just making sure the dust settles first | 18:56 |
| * mordred throws a handful of dust at fungi | 18:57 | |
| fungi | i have plenty already, thanks. the contractors started replacing my hvac system bright and early this morning | 18:58 |
| fungi | i didn't realize these things cost as much as a new mid-size automobile | 18:58 |
| *** jamesmcarthur_ has joined #openstack-infra | 18:59 | |
| *** jamesmcarthur has quit IRC | 19:01 | |
| *** rlandy is now known as rlandy|brb | 19:01 | |
| mordred | fungi: they are oe of the least fun things to replace | 19:07 |
| mordred | you don't even get a new toy to play with when they're done | 19:08 |
| *** sshnaidm|bbl is now known as sshnaidm | 19:09 | |
| fungi | well, we get a couple of fancy new multi-zone touchscreen thermostats | 19:14 |
| fungi | (in theory at least, i mean i saw them included on the quote anyway) | 19:15 |
| fungi | also new ductwork i could crawl around inside if i lost a few lbs | 19:16 |
| *** ralonsoh has quit IRC | 19:22 | |
| *** rlandy|brb is now known as rlandy | 19:26 | |
| smcginnis | Hah | 19:32 |
| smcginnis | Thanks fungi and mordred. | 19:32 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 19:40 |
| fungi | turns out the image changes have broken our ability to test that role, however ;) | 19:40 |
| *** jamesmcarthur_ has quit IRC | 19:45 | |
| *** jamesmcarthur has joined #openstack-infra | 19:46 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 19:46 |
| AJaeger | elod: oslo.messaging is more complex, hope I got it now. fungi, could you review the change, please? https://review.opendev.org/736248 problem is that for amqp1 testing, it needs to install qdrouterd with needs another repo to be added for Ubuntu... | 19:47 |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 19:47 |
| *** ociuhandu has joined #openstack-infra | 19:52 | |
| *** eolivare_ has joined #openstack-infra | 19:55 | |
| *** eolivare_ has quit IRC | 19:58 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add ensure pre-run policy to ansible-lint https://review.opendev.org/736367 | 20:01 |
| *** rlandy_ has joined #openstack-infra | 20:09 | |
| *** mnencia_ has joined #openstack-infra | 20:11 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: phoronix-test-suite: extract ensure- role from run playbook https://review.opendev.org/736368 | 20:11 |
| *** ktsuyuzaki has joined #openstack-infra | 20:11 | |
| *** Lucas_Gray has joined #openstack-infra | 20:11 | |
| *** ktsuyuzaki is now known as kota__ | 20:12 | |
| *** tobberydberg_ has joined #openstack-infra | 20:12 | |
| *** tosky_ has joined #openstack-infra | 20:12 | |
| *** tosky has quit IRC | 20:13 | |
| *** tosky_ is now known as tosky | 20:13 | |
| *** irclogbot_3 has quit IRC | 20:14 | |
| *** JasonF has joined #openstack-infra | 20:14 | |
| *** Wasaac_ has joined #openstack-infra | 20:14 | |
| *** gyee has quit IRC | 20:17 | |
| *** tobberydberg has quit IRC | 20:18 | |
| *** rlandy has quit IRC | 20:18 | |
| *** zxiiro has quit IRC | 20:18 | |
| *** masayukig has quit IRC | 20:18 | |
| *** mnencia has quit IRC | 20:18 | |
| *** mgoddard has quit IRC | 20:18 | |
| *** amorin has quit IRC | 20:18 | |
| *** JayF has quit IRC | 20:18 | |
| *** kota_ has quit IRC | 20:18 | |
| *** Wasaac has quit IRC | 20:18 | |
| *** bradm has quit IRC | 20:18 | |
| *** mnencia_ is now known as mnencia | 20:18 | |
| *** rlandy__ has joined #openstack-infra | 20:18 | |
| *** Wasaac_ is now known as Wasaac | 20:19 | |
| *** rlandy_ has quit IRC | 20:19 | |
| *** amorin has joined #openstack-infra | 20:20 | |
| *** irclogbot_3 has joined #openstack-infra | 20:20 | |
| *** mgoddard has joined #openstack-infra | 20:21 | |
| *** masayukig has joined #openstack-infra | 20:24 | |
| *** zxiiro has joined #openstack-infra | 20:25 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 20:27 |
| *** gyee has joined #openstack-infra | 20:31 | |
| *** jamesmcarthur has quit IRC | 20:31 | |
| *** jamesmcarthur has joined #openstack-infra | 20:32 | |
| *** lxkong has quit IRC | 20:35 | |
| *** srwilkers has quit IRC | 20:35 | |
| *** jamesmcarthur has quit IRC | 20:37 | |
| *** howell has quit IRC | 20:37 | |
| *** rpioso has quit IRC | 20:38 | |
| *** davecore has quit IRC | 20:38 | |
| *** cjohnston has quit IRC | 20:38 | |
| *** rlandy__ is now known as rlandy | 20:39 | |
| *** csatari has quit IRC | 20:39 | |
| *** cjohnston has joined #openstack-infra | 20:40 | |
| *** knikolla has quit IRC | 20:40 | |
| *** davecore has joined #openstack-infra | 20:40 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 20:42 |
| *** knikolla has joined #openstack-infra | 20:42 | |
| *** tonyb[m] has quit IRC | 20:43 | |
| *** cjohnston has quit IRC | 20:46 | |
| *** ociuhandu has quit IRC | 20:48 | |
| *** PrinzElvis has quit IRC | 20:49 | |
| *** priteau has quit IRC | 20:49 | |
| *** davecore has quit IRC | 20:51 | |
| *** knikolla has quit IRC | 20:52 | |
| *** PrinzElvis has joined #openstack-infra | 20:52 | |
| *** PrinzElvis has quit IRC | 20:57 | |
| *** diablo_rojo_phon has joined #openstack-infra | 20:59 | |
| *** jamesmcarthur has joined #openstack-infra | 21:04 | |
| *** JasonF is now known as JayF | 21:06 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:07 |
| *** lxkong has joined #openstack-infra | 21:09 | |
| *** rfolco|rover has quit IRC | 21:12 | |
| *** noonedeadpunk has quit IRC | 21:12 | |
| *** jamesmcarthur has quit IRC | 21:13 | |
| openstackgerrit | Ahmad Mahmoudi proposed zuul/zuul-jobs master: (fix) Changed pip to pip3 https://review.opendev.org/736379 | 21:16 |
| *** lxkong has quit IRC | 21:16 | |
| *** ociuhandu has joined #openstack-infra | 21:19 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add ensure pre-run policy to ansible-lint https://review.opendev.org/736367 | 21:19 |
| *** xek has quit IRC | 21:20 | |
| *** ociuhandu has quit IRC | 21:24 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 21:31 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:34 |
| *** davecore has joined #openstack-infra | 21:34 | |
| *** lxkong has joined #openstack-infra | 21:34 | |
| *** srwilkers has joined #openstack-infra | 21:34 | |
| *** howell has joined #openstack-infra | 21:35 | |
| *** PrinzElvis has joined #openstack-infra | 21:35 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:35 |
| *** rpioso has joined #openstack-infra | 21:37 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:37 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 21:37 |
| *** tonyb[m] has joined #openstack-infra | 21:38 | |
| *** csatari has joined #openstack-infra | 21:38 | |
| *** knikolla has joined #openstack-infra | 21:40 | |
| *** slaweq has quit IRC | 21:41 | |
| *** cjohnston has joined #openstack-infra | 21:41 | |
| *** gshippey_ has quit IRC | 21:42 | |
| *** jamesmcarthur has joined #openstack-infra | 21:50 | |
| *** hashar has quit IRC | 21:52 | |
| *** jamesmcarthur has quit IRC | 21:54 | |
| *** tosky has quit IRC | 21:56 | |
| *** tosky has joined #openstack-infra | 21:56 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 21:58 |
| *** rlandy is now known as rlandy|bbl | 21:59 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 21:59 |
| *** matt_kosut has quit IRC | 22:04 | |
| *** matt_kosut has joined #openstack-infra | 22:05 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Add option to install tox into a path location https://review.opendev.org/736330 | 22:05 |
| *** matt_kosut has quit IRC | 22:09 | |
| mordred | smcginnis: ^^ | 22:11 |
| mordred | clarkb: ^^ if you have a sec, https://review.opendev.org/#/c/736334 could use a +A | 22:12 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Fix ansible-lint rules tests https://review.opendev.org/736387 | 22:13 |
| clarkb | mordred: done | 22:13 |
| mordred | \o/ | 22:15 |
| * mordred is happy that there has been a success for the day | 22:16 | |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Symlink tox into the exec path for release jobs https://review.opendev.org/736334 | 22:24 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add linting rule to enforce no-same-owner policy https://review.opendev.org/727642 | 22:29 |
| *** _erlon_ has joined #openstack-infra | 22:33 | |
| *** rcernin has joined #openstack-infra | 22:42 | |
| smcginnis | Awesome, new release test on its way. | 22:44 |
| *** bradm has joined #openstack-infra | 22:44 | |
| fungi | smcginnis: i expect you're still likely to hit trouble until 736185 merges | 22:45 |
| smcginnis | fungi: Yeah, just saw your comment in the release channel. | 22:45 |
| smcginnis | I thought that one had gone through too. I'll hold off a bit. | 22:45 |
| fungi | it had gone through once, had to be reverted because it broke other users outside opendev, and was reworked in a much safer way | 22:46 |
| smcginnis | That might explain it. | 22:47 |
| smcginnis | One job left. | 22:47 |
| openstackgerrit | Merged zuul/zuul-jobs master: Ensure wheel exists for build-release-python https://review.opendev.org/736185 | 22:48 |
| smcginnis | And then there were none. | 22:49 |
| smcginnis | fungi: Do I need to wait for any background processes to run before everything is in place for a test release? | 22:49 |
| clarkb | smcginnis: not if the changes you need are to zuul jobs, those should update basically as soon as they merge | 22:50 |
| smcginnis | Perfect. Release test on its way then. | 22:50 |
| fungi | yay! | 22:51 |
| *** seongsoocho has joined #openstack-infra | 22:51 | |
| *** gyee has quit IRC | 22:52 | |
| *** tkajinam has joined #openstack-infra | 22:54 | |
| *** Lucas_Gray has quit IRC | 23:07 | |
| *** tosky has quit IRC | 23:09 | |
| *** Lucas_Gray has joined #openstack-infra | 23:22 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!