| opendevreview | Merged openstack/project-config master: Add ubuntu-bionic-32GB https://review.opendev.org/c/openstack/project-config/+/814683 | 00:05 |
|---|---|---|
| opendevreview | Merged openstack/project-config master: infra-package-needs: install latest pip https://review.opendev.org/c/openstack/project-config/+/814677 | 00:22 |
| *** ykarel_ is now known as ykarel | 04:40 | |
| opendevreview | Merged openstack/project-config master: Remove fedora-32 disk image config https://review.opendev.org/c/openstack/project-config/+/795644 | 04:40 |
| *** ysandeep|out is now known as ysandeep | 05:08 | |
| *** ysandeep is now known as ysandeep|afk | 06:34 | |
| *** ysandeep|afk is now known as ysandeep|trng | 07:03 | |
| *** jpena|off is now known as jpena | 07:29 | |
| *** ykarel is now known as ykarel|lunch | 08:00 | |
| *** iurygregory_ is now known as iurygregory | 09:00 | |
| *** ykarel|lunch is now known as ykarel | 10:15 | |
| *** ysandeep|trng is now known as ysandeep|afk | 10:19 | |
| *** rlandy is now known as rlandy|ruck | 10:31 | |
| *** dviroel|rover|afk is now known as dviroel|rover | 11:18 | |
| *** jpena is now known as jpena|lunch | 11:25 | |
| *** jcapitao is now known as jcapitao_lunch | 11:42 | |
| zul | hey guys it seems we are still having problems with the bindep issue from yesterday, any eta on this on the xenial images https://zuul.opendev.org/t/openstack/build/0f2355250c814549b164c0fc6c8f9c5d | 12:13 |
| *** ysandeep|afk is now known as ysandeep | 12:21 | |
| *** jpena|lunch is now known as jpena | 12:24 | |
| frickler | zul: if I look at the build logs, it seems the next build should start in 10 mins, so maybe check again in 2-3h | 12:41 |
| zul | ack | 12:41 |
| *** jcapitao_lunch is now known as jcapitao | 12:43 | |
| fungi | yeah, the current images were uploaded around 21.5 hours ago, so chances are we'll have nodes booting from newly uploaded images in another ~2.5 hours | 12:53 |
| *** timburke_ is now known as timburke | 12:58 | |
| opendevreview | Merged openstack/openstack-zuul-jobs master: Add OpenStack Charm gate jobs without Python 3.5 https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/814126 | 13:10 |
| zaitcev | How do I report an issue to the infra team? Looks like a commit-msg hook is missing on opendev.org, although it's present at review.opendev.org. | 13:36 |
| zaitcev | Here: https://bugs.launchpad.net/swift/+bug/1947862 | 13:36 |
| fungi | zaitcev: opendev.org wouldn't have any need for the gerrit commit-msg hook, gerrit serves that because it's the only system users can push to | 13:40 |
| frickler | zaitcev: what version of git-review are you using? I cannot reproduce this locally | 13:41 |
| zaitcev | fungi: see the bug though. When cloned from opendev.org, git-review constructs a URL for commit-msg which points to opendev.org. It ignores .gitreview in such case. | 13:41 |
| fungi | according to that bug report, it's trying to retrieve the hook from github? | 13:41 |
| fungi | The following command failed with exit code 104 "GET https://github.com/tools/hooks/commit-msg" | 13:42 |
| fungi | the only reason i can think of git-review would do that is if the user created a custom git remote named "gerrit" and pointed it at github | 13:43 |
| fungi | or overriding the gerrit.host setting git-review relies on to be github instead of review.opendev.org | 13:44 |
| fungi | regardless, it looks like someone has misconfigured something badly there | 13:44 |
| zaitcev | Indeed cloning from opendev.org and git-review -s work for me too, but they go straight to ssh:. | 13:45 |
| zaitcev | It says | 13:45 |
| zaitcev | "No remote set, testing ssh://zaitcev@review.opendev.org:29418/openstack/swift.git" | 13:45 |
| zaitcev | and everything is fine then. | 13:45 |
| fungi | yes, git-review uses gerrit's ssh-based socket for git operations by default, though can be configured to use https | 13:46 |
| fungi | maybe the user put the wrong thing in when setting it up to interact with gerrit over https, and told it github was the gerrit server | 13:47 |
| zaitcev | I see. Thanks a lot, guys. I'll update the bug and we'll take it from there. | 13:48 |
| fungi | thanks zaitcev! if they need more help troubleshooting git-review settings, they can always ask in here or on the ml... bug trackers aren't really great places to try to do user support obviously | 13:51 |
| zaitcev | Indeed. | 13:51 |
| zaitcev | Although that's pretty much how I do it at work... Very hard to get in touch with users. | 13:51 |
| clarkb | they may be creating a gerrit remote that isn't gerrit? | 13:52 |
| clarkb | zul: re your email to the starglinx list I think the issue is with python3.5 not python2.7 on older systems | 13:53 |
| fungi | from starlingx's perspective, it's breaking their python 2.7 jobs, but it's really that the bindep install on ubuntu-xenial nodes where they're running those py27 jobs is broken | 13:55 |
| fungi | and really the issue is that the last bindep release broke support for old pip which lacks support for the python_requires metadata by adding an open-ended requirement on something which has dropped python 3.5 support | 13:56 |
| zul | clarkb: https://zuul.opendev.org/t/openstack/build/ead6561991e54b13ad2cc6ebea5a82e4 | 13:58 |
| clarkb | ah I'm a bit surprised that starlingx would run py27 jobs on such an old release. Newer releases do ahve python2.7 too | 13:59 |
| zul | im not, which one should we be using for py27 jobs? | 14:10 |
| clarkb | zul: I think focal still has python2.7 | 14:11 |
| clarkb | (though it may be the last one?) | 14:11 |
| zul | i would have to check | 14:11 |
| *** vishalmanchanda_ is now known as vishalmanchanda | 14:24 | |
| *** andrewbonney_ is now known as andrewbonney | 14:26 | |
| opendevreview | Dr. Jens Harbott proposed openstack/project-config master: Move the daily periodic trigger earlier https://review.opendev.org/c/openstack/project-config/+/814794 | 14:34 |
| *** carloss_ is now known as carloss | 15:00 | |
| opendevreview | Elod Illes proposed openstack/openstack-zuul-jobs master: Remove ocata branch from periodic-stable template https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/814799 | 15:01 |
| afaranha | timburke, fungi Hi, I'm back with the results :D I was able to reproduce the issue on the node you gave me | 15:05 |
| afaranha | I hit the out of space issue, and run 'df' | 15:05 |
| afaranha | (.temp_venv) [zuul@centos-8-ovh-gra1-0027003455 swift]$ df -h | 15:05 |
| afaranha | Filesystem Size Used Avail Use% Mounted on | 15:05 |
| afaranha | devtmpfs 3.8G 0 3.8G 0% /dev | 15:05 |
| afaranha | tmpfs 3.8G 0 3.8G 0% /dev/shm | 15:05 |
| afaranha | tmpfs 3.8G 448K 3.8G 1% /run | 15:05 |
| afaranha | tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup | 15:05 |
| afaranha | tmpfs 777M 0 777M 0% /run/user/1000 | 15:05 |
| afaranha | tmpfs 777M 0 777M 0% /run/user/0 | 15:05 |
| afaranha | It seems that the issue is with /mnt/config | 15:06 |
| afaranha | let me put on the paste.openstack, it hod some entried | 15:06 |
| afaranha | entries* | 15:06 |
| afaranha | https://paste.openstack.org/raw/810105/ | 15:07 |
| clarkb | afaranha: /mnt/config is a RO filesystem supplying openstack metadata to the server. I think it is expected that it be 100% full and your code should never try to write to it | 15:10 |
| clarkb | I raelly doubt that is the issue | 15:10 |
| afaranha | clarkb, I don't know where the issue is then :/ xfstmp is at 5% usage only | 15:11 |
| clarkb | is xfs metadata a different usage pool? | 15:13 |
| clarkb | I know btrfs reports usage poorly to df | 15:14 |
| clarkb | and you have to use btrfs specific tools to actually understand use and do a rebalance to free space that otherwise appears available | 15:14 |
| clarkb | xfs_spaceman and xfs_scrub maybe? | 15:14 |
| afaranha | I have no idea what you`re talking about :P | 15:22 |
| afaranha | clarkb, sorry, I'll need some googling on that matter | 15:22 |
| afaranha | "is xfs metadata a different usage pool?" So the issue here is on the metadata itself, right? | 15:22 |
| clarkb | basically df showing you 5% use may not tell the whole story. I would look into xfs specific tools to verify there is free space | 15:23 |
| clarkb | afaranha:I don't know, I don't know much about your tests | 15:23 |
| timburke | fwiw, it's a swift functional test that's failing with fips mode enabled -- but there's another fips func test job on the same patch that *succeeds* | 15:28 |
| timburke | my best guess is that it *is* some kind of xattr/metadata issue, and not the sort of thing that df would easily identify for us | 15:28 |
| *** ykarel is now known as ykarel|away | 15:31 | |
| fungi | afaranha: the question was whether xfs may have a separate limit on the amount of extended attributes it can store, and maybe that's what's been exceeded | 15:48 |
| timburke | and in particular, whether that limit changes when running under fips mode (since the non-fips equivalent passes) | 15:50 |
| *** jpena is now known as jpena|off | 16:31 | |
| *** ysandeep is now known as ysandeep|away | 16:41 | |
| sean-k-mooney | clarkb: fungi quick question if we wanted to move soemthing form storyborad ot launchpad is there any automation for that | 17:01 |
| clarkb | I'm not aware of any | 17:01 |
| sean-k-mooney | e.g. to move active stories to new lanuchpad bugs and to create the launchpad projects | 17:01 |
| sean-k-mooney | ok | 17:01 |
| clarkb | all of the data is public and there are apis on both sides so you should be able to do it, but I'm not aware of anyone having done so | 17:01 |
| sean-k-mooney | if we just flip the defieniton in porject config can the infra tooling create the launchpad project for us | 17:02 |
| fungi | probably the biggest hurdle will be the lp api likes to choke a lot, fall over, and then you have to wait a while to try again | 17:02 |
| fungi | if you're trying to copy old bugs into it | 17:02 |
| fungi | sean-k-mooney: there is no infra tooling which creates launchpad projects, that's all manual | 17:02 |
| sean-k-mooney | ok | 17:03 |
| clarkb | (and there never was) | 17:03 |
| sean-k-mooney | we are not sure if we will actully move things | 17:03 |
| sean-k-mooney | but basiclaly nova uses launchpad for everythying and placment uses story board | 17:03 |
| sean-k-mooney | a few cycles ago placement came back into the compute deliverbale | 17:03 |
| sean-k-mooney | we have not really been monitoring the plamcent story borad projects and we are debating if we we shoudl move the plamcent project in stroyboard to launchpad | 17:04 |
| sean-k-mooney | and if we need to copy any bugs or not | 17:04 |
| sean-k-mooney | i think right now we are not sure the effort to move is worth it but we get a cople of bugs reported in story board every year | 17:06 |
| sean-k-mooney | but we never really triage them or work on them since we dont see them | 17:07 |
| fungi | sb can be set to notify you by e-mail when a project you're monitoring gets a new story task or someone comments on a story which has a task for it, but yes if the people who are working on placement are doing most of their defect and task tracking in lp then it makes sense to tell people to start filing new reports there | 17:11 |
| clarkb | zul is gone but we expect the py35 jobs are happy again | 17:25 |
| clarkb | (and generally other jobs on xenial) | 17:25 |
| fungi | probably any which started after ~14:00z | 17:27 |
| fungi | 814647 was rechecked at 14:10:36 and worked | 17:27 |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 22:25 | |
| *** dviroel|rover is now known as dviroel|rover|afk | 22:53 | |
| *** dviroel|rover|afk is now known as dviroel|rover|out | 23:16 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!