| *** thorst has joined #openstack-infra | 00:00 | |
| *** salv-orl_ has joined #openstack-infra | 00:01 | |
| *** heyongli has quit IRC | 00:02 | |
| *** heyongli has joined #openstack-infra | 00:02 | |
| *** salv-orlando has quit IRC | 00:04 | |
| *** salv-orl_ has quit IRC | 00:04 | |
| *** llu has left #openstack-infra | 00:05 | |
| *** arif-ali has joined #openstack-infra | 00:05 | |
| *** heyongli has quit IRC | 00:12 | |
| *** Swami has quit IRC | 00:12 | |
| *** heyongli has joined #openstack-infra | 00:13 | |
| *** jpr1 has quit IRC | 00:13 | |
| *** jcooley_ has quit IRC | 00:15 | |
| *** sflanigan has quit IRC | 00:16 | |
| *** gildub has joined #openstack-infra | 00:21 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add a job to build Debian packages https://review.openstack.org/294022 | 00:21 |
|---|---|---|
| *** heyongli has quit IRC | 00:22 | |
| *** gongysh_ has joined #openstack-infra | 00:22 | |
| *** heyongli has joined #openstack-infra | 00:23 | |
| anteaya | clarkb: nice stats | 00:23 |
| *** fawadkhaliq has quit IRC | 00:23 | |
| clarkb | anteaya: in fairness to the other projects I should also probably be asking why boot from volume is so stinking slow in the general case | 00:25 |
| clarkb | but didn't want to get distracted :) | 00:26 |
| *** daemontool_ has joined #openstack-infra | 00:26 | |
| *** BobBall_ has joined #openstack-infra | 00:27 | |
| anteaya | good call | 00:27 |
| *** daemontool has quit IRC | 00:27 | |
| anteaya | one hydra at a time | 00:27 |
| *** BobBall_ is now known as Guest28565 | 00:27 | |
| *** gordc has joined #openstack-infra | 00:30 | |
| *** BobBall has quit IRC | 00:30 | |
| *** angdraug has quit IRC | 00:31 | |
| *** Guest28565 has quit IRC | 00:31 | |
| *** rkukura has quit IRC | 00:32 | |
| *** BobBall has joined #openstack-infra | 00:32 | |
| *** BobBall is now known as Guest74619 | 00:32 | |
| *** heyongli has quit IRC | 00:33 | |
| *** jpr has joined #openstack-infra | 00:33 | |
| *** heyongli has joined #openstack-infra | 00:33 | |
| *** roxanaghe has quit IRC | 00:34 | |
| *** Qiming has quit IRC | 00:35 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/system-config: Initial commit of the Phabricator puppetry. https://review.openstack.org/289648 | 00:35 |
| *** agentle has joined #openstack-infra | 00:36 | |
| *** ddieterly has joined #openstack-infra | 00:36 | |
| *** amitgandhinz has joined #openstack-infra | 00:37 | |
| *** agentle has quit IRC | 00:40 | |
| *** eliqiao has quit IRC | 00:42 | |
| *** heyongli has quit IRC | 00:43 | |
| *** eliqiao has joined #openstack-infra | 00:43 | |
| *** heyongli has joined #openstack-infra | 00:43 | |
| *** zz_dimtruck is now known as dimtruck | 00:44 | |
| *** sputnik13 has quit IRC | 00:47 | |
| *** fawadkhaliq has joined #openstack-infra | 00:47 | |
| *** hichihara has quit IRC | 00:49 | |
| *** gongysh_ has quit IRC | 00:50 | |
| *** amitgandhinz has quit IRC | 00:50 | |
| *** tqtran has quit IRC | 00:51 | |
| *** heyongli has quit IRC | 00:53 | |
| *** sflanigan has joined #openstack-infra | 00:53 | |
| *** heyongli has joined #openstack-infra | 00:53 | |
| *** dims has joined #openstack-infra | 00:57 | |
| *** pvaneck has quit IRC | 01:00 | |
| *** bpokorny_ has joined #openstack-infra | 01:00 | |
| *** rossella_s has quit IRC | 01:03 | |
| *** bpokorny has quit IRC | 01:03 | |
| *** sridhar_ram has quit IRC | 01:03 | |
| *** heyongli has quit IRC | 01:03 | |
| *** rossella_s has joined #openstack-infra | 01:04 | |
| *** heyongli has joined #openstack-infra | 01:04 | |
| *** nadya has joined #openstack-infra | 01:04 | |
| *** sridhar_ram has joined #openstack-infra | 01:04 | |
| *** kzaitsev_mb has quit IRC | 01:04 | |
| *** bpokorny_ has quit IRC | 01:05 | |
| *** fawadkhaliq has quit IRC | 01:05 | |
| *** zhurong has joined #openstack-infra | 01:06 | |
| *** fawadkhaliq has joined #openstack-infra | 01:06 | |
| *** jpr has quit IRC | 01:06 | |
| *** sdake has quit IRC | 01:06 | |
| *** sdake has joined #openstack-infra | 01:08 | |
| *** sridhar_ram has quit IRC | 01:09 | |
| *** nadya has quit IRC | 01:09 | |
| *** Sukhdev has quit IRC | 01:09 | |
| *** Qiming has joined #openstack-infra | 01:10 | |
| *** mtanino has quit IRC | 01:10 | |
| *** salv-orlando has joined #openstack-infra | 01:10 | |
| SamYaple | fungi: did you send out the final round of ATCs? | 01:10 |
| *** Jeffrey4l__ has quit IRC | 01:11 | |
| *** sdake_ has joined #openstack-infra | 01:11 | |
| prometheanfire | ianw: one way to find out if they will fight :P | 01:12 |
| SamYaple | clarkb: its not slow to boot from bolume on ceph (with CoW cloning) | 01:12 |
| SamYaple | i think its slow on others because nova pulls glance image then pushes image into swift | 01:13 |
| SamYaple | but its been a while since ive not used ceph backend | 01:13 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add pkg-map to pip-and-virtualenv element https://review.openstack.org/294973 | 01:13 |
| *** heyongli has quit IRC | 01:14 | |
| *** heyongli has joined #openstack-infra | 01:14 | |
| *** MarkAtwood has joined #openstack-infra | 01:14 | |
| *** sdake has quit IRC | 01:14 | |
| clarkb | SamYaple: I am not sure how it all works, I know glance uses swift as the backend and cinder users lvm | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: WIP: Add a OVB te-broker server https://review.openstack.org/295243 | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Add mirror server https://review.openstack.org/238414 | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Use the mirror server for the centos image and git repos https://review.openstack.org/285257 | 01:16 |
| clarkb | SamYaple: but it can take over 2 minutes to run that single test and we run it multiple times using different apis | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Use instack-qcow2 from cache if available and appropriate https://review.openstack.org/285259 | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Upload the instack qcow image after periodic jobs https://review.openstack.org/285258 | 01:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 01:16 |
| prometheanfire | ianw: ^ that should work for you | 01:16 |
| SamYaple | nova is the tied in though, i dont think it goes glance -> cinder, it goes glance -> nova -> cinder | 01:16 |
| ianw | prometheanfire: have you tested it? because ci won't go down that path | 01:16 |
| SamYaple | clarkb: but with ceph CoW its glance-pool copy on write to cinder pool, basically instant | 01:16 |
| prometheanfire | what's the test | 01:17 |
| *** mriedem has quit IRC | 01:17 | |
| prometheanfire | don't understand why they would fight | 01:17 |
| fungi | SamYaple: yes, final batch was sent on march 3 | 01:17 |
| SamYaple | fungi: was the cutoff to contribute march 3 as well? | 01:17 |
| SamYaple | or earlier | 01:17 |
| clarkb | SamYaple: sure I know ceph is faster (we have that data too :) ), but I would expect it to not take 2 minutes when booting from not volume is much faster | 01:17 |
| clarkb | speaking of I need to go address mordred's comments about mirroring that ceph apt repo | 01:18 |
| fungi | SamYaple: cutoff was whatever time i ran the query on march 3 really, though i told everyone commits up to or on march 2 just to be safe | 01:18 |
| SamYaple | clarkb: yea we have a few things to mirror from kolla | 01:18 |
| SamYaple | fungi: ok ill check to see if the person got it | 01:18 |
| prometheanfire | git-pip-py and get_pip_py are not defined anywhere | 01:19 |
| clarkb | skimming the devstack logs the only thing I see as sucking up more than 10 seconds at a time is we create multiple tempest venvs | 01:19 |
| clarkb | together thats almost a minute looks like | 01:19 |
| prometheanfire | before they could have conflicted, but now the source method and the source-fetch are named the same | 01:19 |
| prometheanfire | so should work fine | 01:19 |
| prometheanfire | my first patch could have failed | 01:19 |
| SamYaple | clarkb: these are the repos we have to mirror in infra from kolla to have all packages mirrored https://github.com/openstack/kolla/blob/master/tools/setup_gate.sh#L40 | 01:19 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: [WIP] Support py34 functional tests for Oslo.Messaging https://review.openstack.org/295531 | 01:20 |
| clarkb | SamYaple: so forgive me for being silly aobut this, but why mariadb on ubuntu? | 01:20 |
| *** apoorvad has quit IRC | 01:20 | |
| SamYaple | clarkb: mariadb10 isnt in the 14.04 repos | 01:20 |
| clarkb | ya use mysql | 01:20 |
| clarkb | like everyone else | 01:20 |
| SamYaple | no galera then | 01:20 |
| clarkb | you can galera with mysql | 01:21 |
| clarkb | ... | 01:21 |
| SamYaple | not with the version in the repos | 01:21 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 01:21 | |
| SamYaple | needs a patched wsrep | 01:21 |
| clarkb | wat tripleo was doing it | 01:21 |
| SamYaple | mariadb10 | 01:21 |
| SamYaple | thats what they were using i believe | 01:21 |
| clarkb | pretty sure they werent | 01:21 |
| clarkb | they probably are now because they stopped supporting ubuntu | 01:21 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 01:22 | |
| SamYaple | im pretty sure they were. i rememebr it came up in a heat bug a while back | 01:22 |
| SamYaple | well regardless of if they were or werent, we definetely arent using mysql now | 01:23 |
| SamYaple | mariadb10 -> mysql5.5 doesnt exist | 01:23 |
| SamYaple | rhallisey: can you weigh in here, was tripleo ubuntu using mariadb10? | 01:24 |
| *** heyongli has quit IRC | 01:24 | |
| SamYaple | or do you recall | 01:24 |
| * SamYaple never used tripleo | 01:24 | |
| *** heyongli has joined #openstack-infra | 01:24 | |
| rhallisey | eh | 01:24 |
| rhallisey | tripleo ubuntu? | 01:24 |
| rhallisey | one sec | 01:25 |
| *** salv-orlando has quit IRC | 01:26 | |
| SamYaple | clarkb: thread of relevance http://lists.openstack.org/pipermail/openstack-dev/2014-June/038952.html | 01:26 |
| SamYaple | but i dont see a resolution | 01:26 |
| rhallisey | ya I dunno | 01:26 |
| SamYaple | clarkb: ahh here it is https://review.openstack.org/#/c/90134/37/elements/mysql/install.d/10-mysql they were using percona | 01:27 |
| SamYaple | _NOT_ the built in mysql (which doesnt support galera) | 01:27 |
| *** gongysh_ has joined #openstack-infra | 01:27 | |
| mordred | clarkb: word | 01:27 |
| clarkb | SamYaple: ah | 01:27 |
| *** salv-orlando has joined #openstack-infra | 01:28 | |
| clarkb | SamYaple: anyways we should have the ceph hammer mirror up soonish | 01:29 |
| clarkb | assuming that I don't completely fail at reprepro which is now my least favorite piece of software to configure | 01:29 |
| clarkb | (it seems to assume I actually understand how apt repos work) | 01:29 |
| clarkb | (does anyone?) | 01:29 |
| SamYaple | clarkb: awesome! will you also be mirroring infernalis and/or jewel (when jewel branches) | 01:29 |
| SamYaple | i know apt repos! | 01:29 |
| prometheanfire | ianw: I am going to run a build with and without DIB_INSTALLTYPE_pip_and_virtualenv=package | 01:29 |
| prometheanfire | DIB_INSTALLTYPE_pip_and_virtualenv=package is running now | 01:29 |
| SamYaple | that was a previous life clarkb | 01:29 |
| SamYaple | a wonder ful previous life | 01:30 |
| clarkb | SamYaple: I was only asked to do hammer since that is what people are using. My guess is we will add in the other versions as they appear and people switch to them | 01:30 |
| SamYaple | clarkb: awesome. hammer is what kolla has | 01:30 |
| SamYaple | will be jewel when jewel releases (end-of-april ish) but no rush on switchover | 01:31 |
| SamYaple | clarkb: about the other repos kolla uses.... whats the process to get those mirrored too? | 01:31 |
| clarkb | SamYaple: let me get you a link to me chages as they are now, basically you write a new change to add each repo into our reprepro system | 01:32 |
| SamYaple | i will absolutely do the legwork. thanks | 01:32 |
| SamYaple | do you guys do any work with the preseed system for debian? because I am also fluent in that. ive always been a debian guy | 01:33 |
| clarkb | SamYaple: https://review.openstack.org/#/c/293155/ | 01:33 |
| *** Liuqing has joined #openstack-infra | 01:33 | |
| clarkb | we aren't doing much with debain yet. We do build test slave imgaes and can boot test slaves on jessie now though | 01:34 |
| clarkb | but thats less than a week old | 01:34 |
| SamYaple | well debian/ubuntu preseed system is the same | 01:34 |
| *** heyongli has quit IRC | 01:34 | |
| *** heyongli has joined #openstack-infra | 01:34 | |
| SamYaple | i dont think you do, what with DIB and all. but if you do for any baremetal nodes.... | 01:34 |
| clarkb | for baremetal I think plan is to still dib | 01:35 |
| clarkb | was the issue iwth rax onmetal, we can't push an image to rax for that yet (or couldn't at the time not sure if that is now possible) | 01:35 |
| *** rkukura has joined #openstack-infra | 01:36 | |
| SamYaple | ok. well if you end up needing to use it, i did alot with it in the day and im good with the black magic that is partman https://github.com/SamYaple/openstack_libvirt_deploy/blob/master/httpd/preseed/preseed.cfg#L39 | 01:36 |
| clarkb | and we use dib with bifrost to deploy the openstack cloud | 01:36 |
| clarkb | so we know it works | 01:36 |
| SamYaple | tis the way to go | 01:36 |
| SamYaple | no one chooses to use preseed. it is forced upon them | 01:37 |
| clarkb | SamYaple: but ya if you want ot push other changes like 293155 for the other repos that would be great | 01:37 |
| SamYaple | already on it | 01:37 |
| clarkb | SamYaple: then for the jobs to use them my rough plan was to write out an /etc/apt/sources.d/ceph then any job wanting to use ceph would check if that exists and add it if not | 01:37 |
| clarkb | SamYaple: you could do the same for the other repos have a sources.d/mariadb and so on, and if present use it else write your own | 01:37 |
| SamYaple | ok. so this will be the first non-ubuntu official repo? | 01:38 |
| clarkb | yup | 01:38 |
| SamYaple | ok cool that style works for us. | 01:38 |
| SamYaple | i can tweak my gate script to use that | 01:38 |
| *** Daisy has joined #openstack-infra | 01:38 | |
| SamYaple | because i have to inject that stuff into the containers | 01:38 |
| *** Daisy has quit IRC | 01:38 | |
| prometheanfire | ianw: would you rather me test with DIB_INSTALLTYPE_pip_and_virtualenv=source or unset? | 01:39 |
| *** Daisy has joined #openstack-infra | 01:39 | |
| prometheanfire | it should be the same either way | 01:39 |
| SamYaple | clarkb: we had a 9m build-and-deploy-and-boot-vm gate the other day. its mostly 10m for ubuntu. these mirrors should bring us down to sub 10m all the time | 01:40 |
| clarkb | SamYaple: nice, we do also cache a lot of stuff in the apt cache which you could bind mount into your container | 01:40 |
| clarkb | s | 01:40 |
| clarkb | SamYaple: I bet that would speed stuff up too | 01:40 |
| *** ddieterly is now known as ddieterly[away] | 01:40 | |
| SamYaple | clarkb: oh yea. thats very possible. the apt-cache exists in the image itself? | 01:41 |
| SamYaple | i can probably bake that in, yea | 01:41 |
| openstackgerrit | Martin André proposed openstack/gertty: Add matchers for external references on launchpad https://review.openstack.org/295592 | 01:42 |
| *** salv-orlando has quit IRC | 01:43 | |
| prometheanfire | testing unset then | 01:43 |
| *** yamahata has quit IRC | 01:43 | |
| *** Daisy has quit IRC | 01:43 | |
| *** akshai has quit IRC | 01:44 | |
| *** heyongli has quit IRC | 01:44 | |
| *** heyongli has joined #openstack-infra | 01:45 | |
| cody-somerville | Anybody have any updates they'd like me to include in the weekly infra-cloud update report? :) | 01:45 |
| *** yamamoto_ has joined #openstack-infra | 01:45 | |
| cody-somerville | This is what I have so far: https://etherpad.openstack.org/p/infra-cloud-update-march-21 | 01:46 |
| *** amitgandhinz has joined #openstack-infra | 01:47 | |
| *** tqtran has joined #openstack-infra | 01:49 | |
| clarkb | SamYaple: yup the apt cache is preseeded so that things installing packages can do so quickly | 01:49 |
| *** rkukura has quit IRC | 01:50 | |
| *** ajmiller has joined #openstack-infra | 01:50 | |
| clarkb | SamYaple: should cover all of the packages expressed by devstack as being required | 01:50 |
| SamYaple | clarkb: looking at it further it doesnt look like we can use that as a build-time cache | 01:50 |
| SamYaple | docker doesnt have build-time volumes yet | 01:50 |
| *** Daisy has joined #openstack-infra | 01:52 | |
| *** tqtran has quit IRC | 01:53 | |
| *** Daisy_ has joined #openstack-infra | 01:53 | |
| prometheanfire | ianw: confirmed that expected behavior is working for both source and package now | 01:55 |
| *** lucas-dinner has quit IRC | 01:55 | |
| *** heyongli has quit IRC | 01:55 | |
| clarkb | SamYaple: thats too bad | 01:55 |
| *** heyongli has joined #openstack-infra | 01:55 | |
| clarkb | SamYaple: we have all the things openstack needs in that cache (if it come sfomr the distro repos) | 01:55 |
| *** ajmiller_ has joined #openstack-infra | 01:55 | |
| prometheanfire | anyone with project-config +w access able to review this? https://review.openstack.org/#/c/293193/ | 01:55 |
| SamYaple | clarkb: im going to play with it, see what i can do. i agree its too bad. im all about easing the load on infra | 01:56 |
| clarkb | SamYaple: we do the same for yum/dnf on centos and fedora as well | 01:56 |
| *** woodster_ has quit IRC | 01:57 | |
| clarkb | SamYaple: not sure if you are running tests on those distros but in theory you can do similar to speed things along there | 01:57 |
| *** Daisy has quit IRC | 01:57 | |
| SamYaple | clarkb: technically i could setup an apt-cacher on the gate machine and prepoulate it, then tell the apt/yum in the containers to run through that.... | 01:57 |
| SamYaple | we do centos and ubntu (and soon oraclelinux) | 01:57 |
| *** ajmiller__ has joined #openstack-infra | 01:58 | |
| *** ajmiller_ has quit IRC | 01:58 | |
| *** ajmiller has quit IRC | 01:58 | |
| jhesketh | prometheanfire: lgtm | 01:59 |
| *** amitgandhinz has quit IRC | 02:00 | |
| *** lucasagomes has joined #openstack-infra | 02:00 | |
| clarkb | SamYaple: but ya anything in the distro repos that devstack declares as a dep should be precached. libvirt, rabbit, mysql, etc etc. Its about a gig of data iirc | 02:00 |
| clarkb | very big lis | 02:00 |
| clarkb | *list | 02:00 |
| SamYaple | clarkb: its cool, apt-cacher only scans it. it doesnt manipulate it | 02:01 |
| SamYaple | so i can potentially use it thorugh the netowrk, if not bind mount it into the container | 02:01 |
| clarkb | SamYaple: and you use the wheel mirror already? we should have one for centos soon I think | 02:02 |
| SamYaple | we do | 02:02 |
| clarkb | pabelanger and ianw were working on getting the centos wheel mirror up | 02:02 |
| SamYaple | clarkb: https://github.com/openstack/kolla/blob/master/tools/setup_gate.sh#L27 | 02:02 |
| SamYaple | when it merges, well consume it | 02:03 |
| SamYaple | we use host /etc/pip.conf for contaienrs | 02:03 |
| clarkb | awesome | 02:03 |
| ianw | clarkb: yep, it's just waiting for the slave to be brought up, but everything is in readyness | 02:03 |
| clarkb | ianw: cool, pabelanger want me to walk you through that tomorrow morning? I have to pop out after the infra meeting but all morning pst should be fine | 02:03 |
| *** baoli has quit IRC | 02:03 | |
| *** baoli has joined #openstack-infra | 02:04 | |
| *** heyongli has quit IRC | 02:05 | |
| *** heyongli has joined #openstack-infra | 02:05 | |
| *** baoli has quit IRC | 02:06 | |
| *** tphummel has quit IRC | 02:08 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting diskimage-builder jobs https://review.openstack.org/293193 | 02:08 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Support py34 functional tests for Oslo.Messaging https://review.openstack.org/295531 | 02:09 |
| *** thorst has quit IRC | 02:10 | |
| prometheanfire | jhesketh: wooo, thanks | 02:10 |
| *** thorst has joined #openstack-infra | 02:11 | |
| prometheanfire | just have two dib reviews out then :D | 02:11 |
| *** knikolla has quit IRC | 02:15 | |
| *** heyongli has quit IRC | 02:15 | |
| *** heyongli has joined #openstack-infra | 02:16 | |
| *** baoli has joined #openstack-infra | 02:17 | |
| *** Daisy_ has quit IRC | 02:18 | |
| *** Daisy has joined #openstack-infra | 02:18 | |
| *** thorst has quit IRC | 02:19 | |
| *** yamamoto_ has quit IRC | 02:23 | |
| *** fawadkhaliq has quit IRC | 02:24 | |
| *** heyongli has quit IRC | 02:25 | |
| *** gordc has quit IRC | 02:26 | |
| *** heyongli has joined #openstack-infra | 02:26 | |
| *** _amrith_ is now known as amrith | 02:26 | |
| *** Daisy has quit IRC | 02:28 | |
| *** sbalukoff has quit IRC | 02:28 | |
| *** Daisy has joined #openstack-infra | 02:28 | |
| *** dims has quit IRC | 02:28 | |
| openstackgerrit | Stephanie Miller proposed openstack/diskimage-builder: ironic-agent postinst fails on systemd with no iptables https://review.openstack.org/295597 | 02:30 |
| *** rhallisey has quit IRC | 02:33 | |
| *** _ari_ has joined #openstack-infra | 02:33 | |
| *** baoli_ has joined #openstack-infra | 02:33 | |
| *** _ari_ has quit IRC | 02:35 | |
| *** heyongli has quit IRC | 02:36 | |
| *** heyongli has joined #openstack-infra | 02:36 | |
| *** baoli has quit IRC | 02:37 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Make all configuration in-repo configuration https://review.openstack.org/268386 | 02:38 |
| jeblair | jhesketh: thanks for the catch on the os.walk thing ^ | 02:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove status_url option https://review.openstack.org/291960 | 02:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Sever the forward reference from connections to their users https://review.openstack.org/291855 | 02:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove scheduler parameter from connection registry https://review.openstack.org/291961 | 02:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Move gerrit logic from source to connection https://review.openstack.org/291854 | 02:39 |
| jeblair | rebases 6 | 02:39 |
| jeblair | ^ | 02:39 |
| *** salv-orlando has joined #openstack-infra | 02:39 | |
| jhesketh | jeblair: no worries.. do you want me to merge those in like we did last time (ie without making them perfect or getting 2*+2's | 02:40 |
| jhesketh | ) | 02:40 |
| jeblair | jhesketh: yeah, i think we're still in that phase | 02:40 |
| *** ddieterly[away] has quit IRC | 02:40 | |
| jhesketh | okay I'll take a look at the new ps's and +w them | 02:40 |
| jhesketh | jeblair: btw, if you have time I've added some more comments to the AFS discussion (https://review.openstack.org/#/c/269928/).. not sure if we should move some of it on list | 02:41 |
| *** Daisy has quit IRC | 02:42 | |
| *** Daisy has joined #openstack-infra | 02:42 | |
| jeblair | jhesketh: ah, thanks. it might be useful to take it to the list, but i'll respond in the review now; we can see where we are after that | 02:43 |
| *** mylu has quit IRC | 02:43 | |
| jhesketh | jeblair: sounds good, thanks :-) | 02:44 |
| *** asettle has quit IRC | 02:44 | |
| *** mylu has joined #openstack-infra | 02:46 | |
| *** heyongli has quit IRC | 02:46 | |
| *** heyongli has joined #openstack-infra | 02:46 | |
| *** mylu has quit IRC | 02:48 | |
| *** mylu has joined #openstack-infra | 02:48 | |
| *** akshai has joined #openstack-infra | 02:49 | |
| *** salv-orlando has quit IRC | 02:50 | |
| jhesketh | jeblair: with 268386, why not use os.path.relpath? | 02:50 |
| *** esker has quit IRC | 02:50 | |
| jeblair | jhesketh: no reason | 02:51 |
| jhesketh | worth changing, or too nitpicky? | 02:51 |
| *** ybathia has joined #openstack-infra | 02:51 | |
| jeblair | jhesketh: you want to change it in a followup? :) | 02:51 |
| jhesketh | jeblair: yep, will do | 02:52 |
| *** ybathia_ has joined #openstack-infra | 02:52 | |
| jhesketh | jeblair: ah, it fails pep8.. would you like me to re-roll it for you? | 02:53 |
| *** akshai has quit IRC | 02:53 | |
| jeblair | jhesketh: i'll fix it | 02:53 |
| jhesketh | okay | 02:54 |
| jeblair | except, here's the thing | 02:54 |
| jeblair | i'm going to fix it by removing those comments | 02:54 |
| jeblair | which i'm not happy about | 02:54 |
| jhesketh | that's fine, it makes sense to me... although I would suggest if you are editing that to use something other than absolute as it's technically not absolute on the filesystem | 02:55 |
| jhesketh | filepath makes sense to me | 02:55 |
| jeblair | jhesketh: yeah, i don't know what else to use instead | 02:55 |
| jeblair | "long" | 02:55 |
| jhesketh | (along with relative_filepath as it's obvious relative to source) | 02:55 |
| *** rfolco has quit IRC | 02:55 | |
| *** notnownikki has quit IRC | 02:56 | |
| *** ybathia has quit IRC | 02:56 | |
| *** ybathia_ is now known as ybathia | 02:56 | |
| *** heyongli has quit IRC | 02:56 | |
| *** asettle has joined #openstack-infra | 02:56 | |
| *** amitgandhinz has joined #openstack-infra | 02:56 | |
| *** heyongli has joined #openstack-infra | 02:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Make all configuration in-repo configuration https://review.openstack.org/268386 | 02:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove status_url option https://review.openstack.org/291960 | 02:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Sever the forward reference from connections to their users https://review.openstack.org/291855 | 02:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove scheduler parameter from connection registry https://review.openstack.org/291961 | 02:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Move gerrit logic from source to connection https://review.openstack.org/291854 | 02:57 |
| *** fawadkhaliq has joined #openstack-infra | 03:00 | |
| jeblair | jhesketh: replied on afs spec | 03:03 |
| jhesketh | jeblair: cool ,thanks | 03:03 |
| jeblair | jhesketh: thank you. i need to run; can chat more tomorrow. | 03:03 |
| jhesketh | no worries, have a good evening :-) | 03:04 |
| *** sridhar_ram has joined #openstack-infra | 03:05 | |
| *** baoli_ has quit IRC | 03:05 | |
| *** baoli has joined #openstack-infra | 03:06 | |
| *** heyongli has quit IRC | 03:06 | |
| *** salv-orlando has joined #openstack-infra | 03:07 | |
| *** heyongli has joined #openstack-infra | 03:07 | |
| *** sdake_ is now known as sdake | 03:08 | |
| *** amitgandhinz has quit IRC | 03:10 | |
| *** akshai has joined #openstack-infra | 03:10 | |
| openstackgerrit | Merged openstack-infra/zuul: Make all configuration in-repo configuration https://review.openstack.org/268386 | 03:12 |
| *** yamamoto has joined #openstack-infra | 03:14 | |
| *** heyongli has quit IRC | 03:17 | |
| *** baoli has quit IRC | 03:17 | |
| *** thorst has joined #openstack-infra | 03:17 | |
| *** heyongli has joined #openstack-infra | 03:17 | |
| *** baoli has joined #openstack-infra | 03:17 | |
| *** mylu has quit IRC | 03:19 | |
| *** bpokorny has joined #openstack-infra | 03:19 | |
| *** links has joined #openstack-infra | 03:19 | |
| *** tphummel has joined #openstack-infra | 03:21 | |
| *** baoli has quit IRC | 03:22 | |
| *** baoli has joined #openstack-infra | 03:23 | |
| *** ajmiller__ has quit IRC | 03:23 | |
| *** salv-orlando has quit IRC | 03:24 | |
| *** thorst has quit IRC | 03:24 | |
| *** IlyaG has quit IRC | 03:27 | |
| *** heyongli has quit IRC | 03:27 | |
| *** heyongli has joined #openstack-infra | 03:27 | |
| openstackgerrit | Merged openstack-infra/zuul: Move gerrit logic from source to connection https://review.openstack.org/291854 | 03:28 |
| *** yuanying has quit IRC | 03:28 | |
| *** ayoung has quit IRC | 03:29 | |
| *** Daisy has quit IRC | 03:34 | |
| *** Daisy has joined #openstack-infra | 03:34 | |
| *** bpokorny has quit IRC | 03:34 | |
| *** roxanaghe has joined #openstack-infra | 03:35 | |
| *** heyongli has quit IRC | 03:37 | |
| *** heyongli has joined #openstack-infra | 03:37 | |
| openstackgerrit | Merged openstack-infra/zuul: Sever the forward reference from connections to their users https://review.openstack.org/291855 | 03:38 |
| openstackgerrit | Merged openstack-infra/zuul: Remove status_url option https://review.openstack.org/291960 | 03:38 |
| openstackgerrit | Merged openstack-infra/zuul: Remove scheduler parameter from connection registry https://review.openstack.org/291961 | 03:38 |
| *** yuanying has joined #openstack-infra | 03:38 | |
| *** Daisy has quit IRC | 03:39 | |
| *** achanda has quit IRC | 03:43 | |
| *** yamahata has joined #openstack-infra | 03:43 | |
| *** dimtruck is now known as zz_dimtruck | 03:44 | |
| *** amrith is now known as _amrith_ | 03:44 | |
| *** MarkAtwood has quit IRC | 03:44 | |
| *** Sukhdev has joined #openstack-infra | 03:47 | |
| *** SamYaple is now known as contributed_this | 03:47 | |
| *** heyongli has quit IRC | 03:47 | |
| *** contributed_this is now known as SamYaple | 03:48 | |
| *** heyongli has joined #openstack-infra | 03:48 | |
| *** tqtran has joined #openstack-infra | 03:50 | |
| *** roxanaghe has quit IRC | 03:50 | |
| *** roxanaghe has joined #openstack-infra | 03:54 | |
| *** sridhar_ram1 has joined #openstack-infra | 03:57 | |
| *** heyongli has quit IRC | 03:58 | |
| *** heyongli has joined #openstack-infra | 03:58 | |
| *** tqtran has quit IRC | 03:58 | |
| *** sridhar_ram has quit IRC | 03:59 | |
| *** baoli has quit IRC | 04:03 | |
| *** yuanying has quit IRC | 04:04 | |
| *** akshai has quit IRC | 04:06 | |
| *** yuanying has joined #openstack-infra | 04:06 | |
| *** amitgandhinz has joined #openstack-infra | 04:06 | |
| *** MarkAtwood has joined #openstack-infra | 04:07 | |
| *** heyongli has quit IRC | 04:08 | |
| *** moravec has quit IRC | 04:08 | |
| *** heyongli has joined #openstack-infra | 04:08 | |
| *** chlong|wfh has quit IRC | 04:12 | |
| *** sbalukoff has joined #openstack-infra | 04:12 | |
| *** rkukura has joined #openstack-infra | 04:13 | |
| *** cj has quit IRC | 04:13 | |
| *** cj has joined #openstack-infra | 04:15 | |
| *** heyongli has quit IRC | 04:18 | |
| *** heyongli has joined #openstack-infra | 04:18 | |
| *** jcooley_ has joined #openstack-infra | 04:19 | |
| *** ybathia has quit IRC | 04:19 | |
| *** rlandy has quit IRC | 04:20 | |
| *** amitgandhinz has quit IRC | 04:20 | |
| *** gongysh_ has quit IRC | 04:20 | |
| *** Sukhdev has quit IRC | 04:21 | |
| *** Sukhdev has joined #openstack-infra | 04:21 | |
| *** fawadkhaliq has quit IRC | 04:22 | |
| *** fawadkhaliq has joined #openstack-infra | 04:22 | |
| *** salv-orlando has joined #openstack-infra | 04:23 | |
| *** jcooley_ has quit IRC | 04:24 | |
| *** treaki__ has joined #openstack-infra | 04:24 | |
| *** thorst has joined #openstack-infra | 04:24 | |
| *** chlong has joined #openstack-infra | 04:25 | |
| *** treaki_ has quit IRC | 04:27 | |
| *** Nikolay_St has quit IRC | 04:27 | |
| *** IlyaG has joined #openstack-infra | 04:27 | |
| *** heyongli has quit IRC | 04:28 | |
| *** heyongli has joined #openstack-infra | 04:29 | |
| *** thorst has quit IRC | 04:29 | |
| *** salv-orlando has quit IRC | 04:30 | |
| *** salv-orlando has joined #openstack-infra | 04:32 | |
| *** achanda has joined #openstack-infra | 04:32 | |
| *** IlyaG has quit IRC | 04:32 | |
| *** nadya has joined #openstack-infra | 04:37 | |
| *** heyongli has quit IRC | 04:39 | |
| *** heyongli has joined #openstack-infra | 04:39 | |
| *** roxanaghe has quit IRC | 04:40 | |
| *** nadya has quit IRC | 04:41 | |
| *** roxanaghe has joined #openstack-infra | 04:42 | |
| *** heyongli has quit IRC | 04:49 | |
| *** heyongli has joined #openstack-infra | 04:49 | |
| *** salv-orlando has quit IRC | 04:52 | |
| *** daemontool_ has quit IRC | 04:53 | |
| *** daemontool__ has joined #openstack-infra | 04:53 | |
| *** tqtran has joined #openstack-infra | 04:55 | |
| *** netsin has quit IRC | 04:57 | |
| *** roxanaghe has quit IRC | 04:59 | |
| *** salv-orlando has joined #openstack-infra | 04:59 | |
| *** heyongli has quit IRC | 04:59 | |
| *** tqtran has quit IRC | 04:59 | |
| *** heyongli has joined #openstack-infra | 05:00 | |
| *** Daisy has joined #openstack-infra | 05:01 | |
| *** rossella_s has quit IRC | 05:03 | |
| *** rossella_s has joined #openstack-infra | 05:03 | |
| *** salv-orl_ has joined #openstack-infra | 05:04 | |
| *** zz_dimtruck is now known as dimtruck | 05:05 | |
| *** salv-orl_ has quit IRC | 05:06 | |
| *** netsin has joined #openstack-infra | 05:08 | |
| *** salv-orlando has quit IRC | 05:09 | |
| *** heyongli has quit IRC | 05:09 | |
| *** heyongli has joined #openstack-infra | 05:10 | |
| sdake | anyone round to help with a launchpad bug tracking issue | 05:13 |
| sdake | round/around | 05:13 |
| *** roxanaghe has joined #openstack-infra | 05:13 | |
| sdake | i'd like to track a bug for multiple milestones, and i have seen it done but i dont know which buttons to press | 05:14 |
| sdake | anyone have any pointers? | 05:14 |
| *** amitgandhinz has joined #openstack-infra | 05:16 | |
| *** tphummel has quit IRC | 05:18 | |
| *** _ari_ has joined #openstack-infra | 05:19 | |
| *** jcooley_ has joined #openstack-infra | 05:20 | |
| *** heyongli has quit IRC | 05:20 | |
| *** heyongli has joined #openstack-infra | 05:20 | |
| lifeless | sdake: you can only do that when each milestone is on a different 'series' | 05:22 |
| *** gongysh_ has joined #openstack-infra | 05:23 | |
| sdake | yup - its called nomination, i just dont see the nomination button | 05:23 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1560298 | 05:24 |
| openstack | Launchpad bug 1560298 in kolla "Make heka collect libvirtd.log in nova_libvirt container" [High,In progress] - Assigned to Vikram Hosakote (vhosakot) | 05:24 |
| sdake | i want to nominate this for mitaka-rc2 and newton n1 | 05:24 |
| *** _ari_ has quit IRC | 05:24 | |
| sdake | or mitaka/liberty series | 05:24 |
| lifeless | so, if you have more privileges, you dont' get nominate, instead you get target | 05:24 |
| sdake | rathre mitaka/newton | 05:24 |
| lifeless | if you don't have enough, you don't get either butter | 05:24 |
| lifeless | button | 05:24 |
| *** jcooley_ has quit IRC | 05:25 | |
| *** dimtruck is now known as zz_dimtruck | 05:25 | |
| sdake | i've got it thanks | 05:25 |
| *** thorst has joined #openstack-infra | 05:26 | |
| *** IlyaG has joined #openstack-infra | 05:28 | |
| *** heyongli has quit IRC | 05:30 | |
| *** amitgandhinz has quit IRC | 05:30 | |
| *** heyongli has joined #openstack-infra | 05:30 | |
| *** tphummel has joined #openstack-infra | 05:30 | |
| *** IlyaG has quit IRC | 05:33 | |
| *** thorst has quit IRC | 05:34 | |
| *** MarkAtwood has quit IRC | 05:38 | |
| *** roxanaghe has quit IRC | 05:39 | |
| *** MarkAtwood has joined #openstack-infra | 05:39 | |
| *** heyongli has quit IRC | 05:40 | |
| *** heyongli has joined #openstack-infra | 05:41 | |
| *** sridhar_ram1 has quit IRC | 05:42 | |
| *** heyongli has quit IRC | 05:50 | |
| *** heyongli has joined #openstack-infra | 05:51 | |
| *** fawadkhaliq has quit IRC | 05:52 | |
| *** harlowja_at_home has quit IRC | 05:52 | |
| *** abregman has joined #openstack-infra | 05:52 | |
| *** MarkAtwood has quit IRC | 05:54 | |
| *** achanda has quit IRC | 05:55 | |
| *** kzaitsev_ws has quit IRC | 06:01 | |
| *** heyongli has quit IRC | 06:01 | |
| *** kzaitsev_ws has joined #openstack-infra | 06:01 | |
| *** heyongli has joined #openstack-infra | 06:01 | |
| *** gongysh_ has quit IRC | 06:02 | |
| *** tphummel has quit IRC | 06:02 | |
| *** roxanaghe has joined #openstack-infra | 06:04 | |
| *** nadya has joined #openstack-infra | 06:05 | |
| *** salv-orlando has joined #openstack-infra | 06:07 | |
| *** Daisy has quit IRC | 06:07 | |
| *** tojuvone has joined #openstack-infra | 06:08 | |
| *** Daisy has joined #openstack-infra | 06:08 | |
| *** nadya has quit IRC | 06:08 | |
| *** roxanaghe has quit IRC | 06:10 | |
| *** heyongli has quit IRC | 06:11 | |
| *** heyongli has joined #openstack-infra | 06:11 | |
| *** zz_dimtruck is now known as dimtruck | 06:17 | |
| openstackgerrit | Merged openstack-infra/project-config: keystone: Use db setup for python jobs https://review.openstack.org/294938 | 06:19 |
| *** jcooley_ has joined #openstack-infra | 06:21 | |
| *** heyongli has quit IRC | 06:21 | |
| *** heyongli has joined #openstack-infra | 06:21 | |
| openstackgerrit | Merged openstack-infra/project-config: ironic,ironic-inspector: Use db setup for python jobs https://review.openstack.org/294939 | 06:24 |
| *** salv-orlando has quit IRC | 06:24 | |
| *** Daisy_ has joined #openstack-infra | 06:24 | |
| *** jcooley_ has quit IRC | 06:25 | |
| *** amitgandhinz has joined #openstack-infra | 06:26 | |
| *** rguillebert has joined #openstack-infra | 06:27 | |
| *** Daisy has quit IRC | 06:27 | |
| *** nikil89_ has joined #openstack-infra | 06:29 | |
| *** nikil89_ is now known as naggappan | 06:29 | |
| *** thorst has joined #openstack-infra | 06:31 | |
| *** heyongli has quit IRC | 06:31 | |
| *** heyongli has joined #openstack-infra | 06:32 | |
| *** Daisy_ has quit IRC | 06:34 | |
| *** Daisy has joined #openstack-infra | 06:34 | |
| *** salv-orlando has joined #openstack-infra | 06:35 | |
| *** thorst has quit IRC | 06:38 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 06:39 |
| *** amitgandhinz has quit IRC | 06:40 | |
| *** salv-orlando has quit IRC | 06:41 | |
| *** heyongli has quit IRC | 06:42 | |
| *** heyongli has joined #openstack-infra | 06:42 | |
| *** Daisy_ has joined #openstack-infra | 06:48 | |
| *** dimtruck is now known as zz_dimtruck | 06:48 | |
| *** Daisy has quit IRC | 06:50 | |
| *** heyongli has quit IRC | 06:52 | |
| *** heyongli has joined #openstack-infra | 06:52 | |
| *** gildub has quit IRC | 06:54 | |
| *** Nikolay_St has joined #openstack-infra | 06:59 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Export magnum setting for image name https://review.openstack.org/295649 | 06:59 |
| *** davidlenwell has quit IRC | 07:01 | |
| *** heyongli has quit IRC | 07:02 | |
| *** heyongli has joined #openstack-infra | 07:02 | |
| *** oanson has quit IRC | 07:03 | |
| *** roxanaghe has joined #openstack-infra | 07:05 | |
| *** Daisy_ has quit IRC | 07:07 | |
| *** Daisy has joined #openstack-infra | 07:07 | |
| vsaienko | infra please help to merge https://review.openstack.org/#/c/252814/. It is job for ironic neutron integration. The patch already has +1 from ironic (jroll) and neutron (dougwig) | 07:08 |
| *** davidlenwell has joined #openstack-infra | 07:09 | |
| *** zz_dimtruck is now known as dimtruck | 07:09 | |
| *** roxanaghe has quit IRC | 07:10 | |
| *** Sukhdev has quit IRC | 07:11 | |
| *** mikelk has joined #openstack-infra | 07:11 | |
| *** heyongli has quit IRC | 07:12 | |
| *** heyongli has joined #openstack-infra | 07:13 | |
| yolanda | good morning | 07:21 |
| AJaeger | yolanda: good morning! | 07:21 |
| yolanda | AJaeger, care reviewing https://review.openstack.org/295649 ? | 07:22 |
| *** Nikolay_St has quit IRC | 07:22 | |
| AJaeger | yolanda: will do. | 07:22 |
| yolanda | thx | 07:22 |
| AJaeger | Could you review these ones, please? https://review.openstack.org/#/q/status:open+project:openstack-infra/project-config+branch:master+topic:bindep | 07:22 |
| AJaeger | Those that are WIP I'd like some more feedback on, the others can merge now... | 07:23 |
| yolanda | i'll do | 07:23 |
| *** heyongli has quit IRC | 07:23 | |
| *** heyongli has joined #openstack-infra | 07:23 | |
| AJaeger | thanks! | 07:23 |
| yolanda | so i reviewed some but not approved, i'll +w then | 07:23 |
| * AJaeger likes to see those in while nobody is around - they cause some UNREGISTERED since we rename jobs and zuul will be faster than jenkins as usual... | 07:24 | |
| AJaeger | yolanda: I can +A those as well myself | 07:24 |
| yolanda | how is jjb performing? i wonder if things could have improved with jenkins version upgrade | 07:26 |
| AJaeger | I think clarkb or jeblair was checking that last night and they didn't see any regression. But I didn't follow. You might want to read backscroll | 07:27 |
| *** tesseract has joined #openstack-infra | 07:28 | |
| *** tesseract is now known as Guest91251 | 07:28 | |
| *** sdake has quit IRC | 07:29 | |
| openstackgerrit | Merged openstack-infra/project-config: manila, murano: Use db setup for python jobs https://review.openstack.org/295034 | 07:32 |
| *** heyongli has quit IRC | 07:33 | |
| *** heyongli has joined #openstack-infra | 07:34 | |
| *** amitgandhinz has joined #openstack-infra | 07:36 | |
| *** thorst has joined #openstack-infra | 07:36 | |
| *** Daisy has quit IRC | 07:37 | |
| openstackgerrit | Merged openstack-infra/project-config: oslo.db: Use db setup for python jobs https://review.openstack.org/294948 | 07:37 |
| *** binbincong has joined #openstack-infra | 07:37 | |
| *** Daisy has joined #openstack-infra | 07:38 | |
| *** Daisy has quit IRC | 07:39 | |
| *** rcernin has joined #openstack-infra | 07:39 | |
| *** jtomasek has joined #openstack-infra | 07:39 | |
| *** Daisy has joined #openstack-infra | 07:39 | |
| openstackgerrit | Merged openstack/diskimage-builder: Note requirement for parted on gentoo hosts https://review.openstack.org/294972 | 07:41 |
| openstackgerrit | Merged openstack-infra/infra-ansible: Add keypairs to infra_config sample and option to use key file https://review.openstack.org/294791 | 07:41 |
| ianw | AJaeger: i wasn't quite sure what to do with -> https://review.openstack.org/#/c/295521/ | 07:43 |
| *** heyongli has quit IRC | 07:43 | |
| *** kushal has joined #openstack-infra | 07:43 | |
| ianw | where's the line between bindep & the infra-extras element in project-config? | 07:43 |
| *** heyongli has joined #openstack-infra | 07:43 | |
| *** thorst has quit IRC | 07:44 | |
| *** Daisy_ has joined #openstack-infra | 07:45 | |
| *** asettle has quit IRC | 07:45 | |
| *** dimtruck is now known as zz_dimtruck | 07:46 | |
| *** salv-orlando has joined #openstack-infra | 07:46 | |
| *** |-paul-| has joined #openstack-infra | 07:47 | |
| AJaeger | ianw: I was confused as well and therefore didn't give a +2 yet ;( | 07:47 |
| openstackgerrit | Merged openstack-infra/project-config: cinder,glance,heat: Use db setup for python jobs https://review.openstack.org/294953 | 07:48 |
| AJaeger | ianw: Let's ask pabelanger and fungi about 295521, hope they can give some guidance on it. | 07:48 |
| *** salv-orlando has quit IRC | 07:48 | |
| *** Daisy has quit IRC | 07:48 | |
| AJaeger | ianw, could you review https://review.openstack.org/295445, https://review.openstack.org/295437 and https://review.openstack.org/295433 , please? Those are db usage changes | 07:49 |
| *** ihrachys has joined #openstack-infra | 07:49 | |
| *** amitgandhinz has quit IRC | 07:49 | |
| *** daemontool__ has quit IRC | 07:50 | |
| openstackgerrit | Merged openstack-infra/project-config: Export magnum setting for image name https://review.openstack.org/295649 | 07:51 |
| *** Kennan has quit IRC | 07:53 | |
| *** salv-orlando has joined #openstack-infra | 07:53 | |
| *** Kennan has joined #openstack-infra | 07:53 | |
| *** mrmartin has joined #openstack-infra | 07:53 | |
| *** heyongli has quit IRC | 07:53 | |
| *** heyongli has joined #openstack-infra | 07:54 | |
| *** flepied1 has joined #openstack-infra | 07:55 | |
| openstackgerrit | Merged openstack-infra/project-config: enable voting for puppet-openstack-cookiecutter jobs https://review.openstack.org/295027 | 07:56 |
| *** asettle has joined #openstack-infra | 07:57 | |
| *** flepied has quit IRC | 07:58 | |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo https://review.openstack.org/295260 | 07:59 |
| *** salv-orlando has quit IRC | 07:59 | |
| *** scheuran has joined #openstack-infra | 08:00 | |
| *** placko_ has joined #openstack-infra | 08:01 | |
| *** asettle has quit IRC | 08:02 | |
| *** flepied1 has quit IRC | 08:02 | |
| *** tqtran has joined #openstack-infra | 08:02 | |
| *** salv-orlando has joined #openstack-infra | 08:03 | |
| *** heyongli has quit IRC | 08:04 | |
| *** heyongli has joined #openstack-infra | 08:04 | |
| *** akaszuba has joined #openstack-infra | 08:05 | |
| *** tqtran has quit IRC | 08:06 | |
| openstackgerrit | Arie Bregman proposed openstack-infra/infra-ansible: Fix tiny styling issue https://review.openstack.org/294781 | 08:08 |
| *** Daisy_ has quit IRC | 08:08 | |
| openstackgerrit | Merged openstack-infra/project-config: New Project - jacket https://review.openstack.org/263500 | 08:09 |
| *** Daisy has joined #openstack-infra | 08:09 | |
| anteaya | abregman: hello | 08:10 |
| anteaya | abregman: did you get the pastes I left you yesterday? | 08:11 |
| *** Nikolay_St has joined #openstack-infra | 08:11 | |
| vsaienko | could you please review https://review.openstack.org/#/c/294128/ it is a first system test job for networking-generic-switch project. | 08:11 |
| *** Daisy has quit IRC | 08:13 | |
| openstackgerrit | huangtianhua proposed openstack/requirements: Bump python-cinderclient to 1.6.0 https://review.openstack.org/295662 | 08:13 |
| *** matrohon has joined #openstack-infra | 08:14 | |
| *** salv-orlando has quit IRC | 08:14 | |
| *** heyongli has quit IRC | 08:14 | |
| anteaya | abregman: I'm off to bed again, I left some pastes in backscroll for you on late friday/early saturday and again yesterday, let me know if you recieve them | 08:14 |
| *** heyongli has joined #openstack-infra | 08:14 | |
| abregman | anteaya: hi. morning. yes, I sent you a message but disconnected shortly after so maybe you didn't receive it. wanted to thank you for the answers. it really helped me and I really appreciate the time you invested in answering on those questions. | 08:16 |
| *** salv-orlando has joined #openstack-infra | 08:16 | |
| abregman | anteaya: I'll take a look again to see that I didn't miss any paste | 08:16 |
| *** markvoelker has quit IRC | 08:16 | |
| *** mvkil has quit IRC | 08:22 | |
| *** jcooley_ has joined #openstack-infra | 08:22 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Bump uc for openstack-doc-tools 0.34 https://review.openstack.org/295665 | 08:23 |
| *** dtardivel has joined #openstack-infra | 08:23 | |
| *** heyongli has quit IRC | 08:24 | |
| *** heyongli has joined #openstack-infra | 08:25 | |
| *** salv-orlando has quit IRC | 08:25 | |
| *** vincentll has joined #openstack-infra | 08:26 | |
| *** jcooley_ has quit IRC | 08:27 | |
| *** ifarkas has joined #openstack-infra | 08:27 | |
| *** arxcruz has joined #openstack-infra | 08:28 | |
| *** binbincong has quit IRC | 08:28 | |
| *** roxanaghe has joined #openstack-infra | 08:30 | |
| *** flepied has joined #openstack-infra | 08:31 | |
| *** zeih has joined #openstack-infra | 08:33 | |
| *** permalac has quit IRC | 08:33 | |
| *** openstackgerrit has quit IRC | 08:33 | |
| *** openstackgerrit has joined #openstack-infra | 08:33 | |
| openstackgerrit | Merged openstack-infra/project-config: storyboard: Use db setup for python jobs https://review.openstack.org/295445 | 08:34 |
| *** daemontool has joined #openstack-infra | 08:34 | |
| *** roxanaghe has quit IRC | 08:34 | |
| *** heyongli has quit IRC | 08:34 | |
| *** heyongli has joined #openstack-infra | 08:35 | |
| *** flepied1 has joined #openstack-infra | 08:35 | |
| *** flepied has quit IRC | 08:35 | |
| openstackgerrit | Merged openstack-infra/project-config: taskflow: Use db setup for python jobs https://review.openstack.org/295437 | 08:36 |
| *** jlanoux has joined #openstack-infra | 08:37 | |
| *** zz_dimtruck is now known as dimtruck | 08:38 | |
| openstackgerrit | Merged openstack-infra/project-config: nodepool: Use db setup for python jobs https://review.openstack.org/295433 | 08:39 |
| *** pcaruana has joined #openstack-infra | 08:40 | |
| *** zeih has quit IRC | 08:40 | |
| *** Daisy has joined #openstack-infra | 08:40 | |
| *** binbincong has joined #openstack-infra | 08:41 | |
| *** _degorenko|afk is now known as degorenko | 08:41 | |
| *** thorst has joined #openstack-infra | 08:42 | |
| *** yaume has joined #openstack-infra | 08:44 | |
| *** Qiming has quit IRC | 08:44 | |
| *** Daisy has quit IRC | 08:45 | |
| *** heyongli has quit IRC | 08:45 | |
| *** heyongli has joined #openstack-infra | 08:45 | |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo https://review.openstack.org/295260 | 08:46 |
| *** amitgandhinz has joined #openstack-infra | 08:46 | |
| *** nmagnezi has joined #openstack-infra | 08:47 | |
| *** Daisy has joined #openstack-infra | 08:48 | |
| *** Qiming has joined #openstack-infra | 08:49 | |
| *** thorst has quit IRC | 08:49 | |
| *** dingyichen has quit IRC | 08:50 | |
| *** jordanP has joined #openstack-infra | 08:52 | |
| openstackgerrit | Arie Bregman proposed openstack-infra/infra-ansible: Add cleanup role https://review.openstack.org/295678 | 08:54 |
| *** heyongli has quit IRC | 08:55 | |
| *** heyongli has joined #openstack-infra | 08:55 | |
| *** ramishra has quit IRC | 08:56 | |
| *** daemontool has quit IRC | 08:57 | |
| *** ramishra has joined #openstack-infra | 08:59 | |
| *** amitgandhinz has quit IRC | 08:59 | |
| *** HeOS has joined #openstack-infra | 09:01 | |
| *** achanda has joined #openstack-infra | 09:02 | |
| *** achanda has quit IRC | 09:02 | |
| *** tqtran has joined #openstack-infra | 09:03 | |
| *** rossella_s has quit IRC | 09:03 | |
| *** rossella_s has joined #openstack-infra | 09:03 | |
| *** doug-fish has quit IRC | 09:04 | |
| *** heyongli has quit IRC | 09:05 | |
| *** heyongli has joined #openstack-infra | 09:06 | |
| *** ihrachys has quit IRC | 09:06 | |
| *** esikache1 has joined #openstack-infra | 09:07 | |
| *** tqtran has quit IRC | 09:07 | |
| *** hashar has joined #openstack-infra | 09:10 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-gerrit: Order of the classes parameters is refactored https://review.openstack.org/294728 | 09:13 |
| *** ihrachys has joined #openstack-infra | 09:15 | |
| *** heyongli has quit IRC | 09:15 | |
| *** heyongli has joined #openstack-infra | 09:16 | |
| *** shardy has joined #openstack-infra | 09:17 | |
| *** derekh has joined #openstack-infra | 09:17 | |
| *** markvoelker has joined #openstack-infra | 09:17 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-cgit: Order of the classes parameters is refactored https://review.openstack.org/295117 | 09:21 |
| *** salv-orlando has joined #openstack-infra | 09:22 | |
| *** e0ne has joined #openstack-infra | 09:22 | |
| *** vgridnev has joined #openstack-infra | 09:23 | |
| *** jcooley_ has joined #openstack-infra | 09:23 | |
| *** jaosorior has joined #openstack-infra | 09:23 | |
| *** sfinucan has joined #openstack-infra | 09:24 | |
| *** dimtruck is now known as zz_dimtruck | 09:26 | |
| *** heyongli has quit IRC | 09:26 | |
| *** heyongli has joined #openstack-infra | 09:26 | |
| *** jcooley_ has quit IRC | 09:28 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-drupal: Order of the classes parameters is refactored https://review.openstack.org/295130 | 09:28 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-infracloud: Order of the classes parameters is refactored https://review.openstack.org/295162 | 09:30 |
| *** sshnaidm has quit IRC | 09:30 | |
| *** placko_ has quit IRC | 09:31 | |
| *** sshnaidm has joined #openstack-infra | 09:31 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-jenkins: Order of the classes parameters is refactored https://review.openstack.org/295171 | 09:34 |
| *** nadya_ has joined #openstack-infra | 09:35 | |
| *** heyongli has quit IRC | 09:36 | |
| *** heyongli has joined #openstack-infra | 09:36 | |
| *** Jeffrey4l has joined #openstack-infra | 09:37 | |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox-neo https://review.openstack.org/295260 | 09:37 |
| *** sshnaidm has quit IRC | 09:37 | |
| *** rpodolyaka has joined #openstack-infra | 09:38 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-meetbot: Indentation of the parameters is refactored https://review.openstack.org/295280 | 09:38 |
| *** kelepirci_ has quit IRC | 09:38 | |
| *** salv-orlando has quit IRC | 09:39 | |
| *** kelepirci_ has joined #openstack-infra | 09:39 | |
| *** dizquierdo has joined #openstack-infra | 09:41 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-openstackci: Order of the classes parameters is refactored https://review.openstack.org/295309 | 09:41 |
| *** Qiming has quit IRC | 09:42 | |
| evgenyl | Good morning, could you please review and merge patch on repo creation https://review.openstack.org/#/c/292765/ there is already a single +2, I need one more. Thanks. | 09:43 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-openstackid: Order of the classes parameters is refactored https://review.openstack.org/295320 | 09:43 |
| *** sshnaidm has joined #openstack-infra | 09:43 | |
| *** claudiub has joined #openstack-infra | 09:43 | |
| *** electrofelix has joined #openstack-infra | 09:44 | |
| *** sdague has joined #openstack-infra | 09:45 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-reviewday: Order of the class parameters is refactored https://review.openstack.org/295357 | 09:45 |
| *** heyongli has quit IRC | 09:46 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-storyboard: Order of the classes parameters is refactored https://review.openstack.org/295372 | 09:46 |
| *** heyongli has joined #openstack-infra | 09:47 | |
| *** thorst has joined #openstack-infra | 09:47 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-zanata: Order of the classes parameters is refactored https://review.openstack.org/295382 | 09:48 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-ansible: Indentation of the class parameters is refactored https://review.openstack.org/294960 | 09:49 |
| *** claudiub|2 has joined #openstack-infra | 09:49 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Reset the ovs nightly https://review.openstack.org/292480 | 09:49 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-iptables: Indentation of the class parameters is refactored https://review.openstack.org/295164 | 09:50 |
| AJaeger | infra-root, any idea what's going on with 293777 ? According to http://status.openstack.org/zuul/ it's for 10 hours on top of check queue ;( There're three more changes below that that are overdue | 09:50 |
| *** sambetts|afk is now known as sambetts | 09:50 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-simpleproxy: Indentation of the class parameter is refactored https://review.openstack.org/295359 | 09:50 |
| *** dims has joined #openstack-infra | 09:51 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config: Add mitaka stable branch jobs https://review.openstack.org/295499 | 09:51 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-unattended_upgrades: Indentation of the class parameter is refactored https://review.openstack.org/295378 | 09:51 |
| yolanda | AJaeger, i can look | 09:51 |
| *** gnuoy_ has joined #openstack-infra | 09:51 | |
| *** claudiub has quit IRC | 09:52 | |
| *** markvoelker has quit IRC | 09:52 | |
| yolanda | gate-grenade-dsvm-neutron: queued .. not sure why is not finding a node | 09:52 |
| openstackgerrit | Arie Bregman proposed openstack-infra/infra-ansible: Support security group creation https://review.openstack.org/295695 | 09:52 |
| *** gildub has joined #openstack-infra | 09:52 | |
| *** zhurong has quit IRC | 09:53 | |
| *** thorst has quit IRC | 09:54 | |
| AJaeger | yolanda: there was a network maintenance window on rax,see https://wiki.openstack.org/wiki/Infrastructure_Status | 09:54 |
| AJaeger | did that cause any trouble? | 09:54 |
| *** esikache1 has quit IRC | 09:55 | |
| *** sshnaidm has quit IRC | 09:55 | |
| *** esikachev has joined #openstack-infra | 09:55 | |
| yolanda | AJaeger, what's type of node needed for that job, you know? | 09:55 |
| *** gnuoy_ has quit IRC | 09:56 | |
| AJaeger | I expect ubuntu-trusty - those jobs should all use it. | 09:56 |
| AJaeger | But let me double check... | 09:56 |
| *** amitgandhinz has joined #openstack-infra | 09:56 | |
| yolanda | it's weird that is not assigning an ubuntu-trusty node while others are running on it | 09:56 |
| yolanda | going to check nodepool logs just in case | 09:56 |
| *** heyongli has quit IRC | 09:56 | |
| AJaeger | https://jenkins02.openstack.org/job/gate-grenade-dsvm-neutron/4578/ - ubuntu-trusty | 09:57 |
| *** heyongli has joined #openstack-infra | 09:57 | |
| AJaeger | yolanda: the network outage that fungi mentioned - see wiki link I gave - might have some relevance here... | 09:57 |
| yolanda | yes i have opened | 09:57 |
| AJaeger | but looking at time, 10h is before that one, so I might be on the wrong track | 09:58 |
| *** gnuoy_ has joined #openstack-infra | 09:58 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config: Add mitaka stable branch jobs https://review.openstack.org/295499 | 09:58 |
| yolanda | what i don't understand is why jobs lower in the queue can get ubuntu-trusty nodes, while this one is queued | 09:59 |
| AJaeger | yolanda: agreed | 09:59 |
| *** Liuqing has quit IRC | 09:59 | |
| *** Hal1 has joined #openstack-infra | 10:00 | |
| yolanda | i see some old ready nodes, going to remove them to regenerate, to see if that helps | 10:00 |
| *** dims has quit IRC | 10:04 | |
| yolanda | AJaeger, i'm seeing errors when connecting to jenkins JenkinsException: Error in request. Possibly authentication failed [500]: Server Error | 10:06 |
| yolanda | on different clouds, that's not good | 10:06 |
| *** sshnaidm has joined #openstack-infra | 10:07 | |
| *** heyongli has quit IRC | 10:07 | |
| *** heyongli has joined #openstack-infra | 10:07 | |
| *** amitgandhinz has quit IRC | 10:09 | |
| *** oanson has joined #openstack-infra | 10:10 | |
| *** Daisy has quit IRC | 10:10 | |
| *** Daisy has joined #openstack-infra | 10:10 | |
| *** pblaho has joined #openstack-infra | 10:12 | |
| *** dims has joined #openstack-infra | 10:12 | |
| yolanda | but is strange because we are getting nodes on jenkins, i wonder if that's happen on a particular master | 10:12 |
| *** tpsilva has joined #openstack-infra | 10:13 | |
| *** jaosorior has quit IRC | 10:15 | |
| *** jaosorior has joined #openstack-infra | 10:15 | |
| *** heyongli has quit IRC | 10:17 | |
| *** heyongli has joined #openstack-infra | 10:17 | |
| *** zz_dimtruck is now known as dimtruck | 10:18 | |
| *** roxanaghe has joined #openstack-infra | 10:18 | |
| yolanda | AJaeger, ok, jenkins03 is failing | 10:20 |
| *** Hal1 has quit IRC | 10:20 | |
| *** Hal1 has joined #openstack-infra | 10:20 | |
| *** akrivoka has joined #openstack-infra | 10:21 | |
| *** roxanaghe has quit IRC | 10:23 | |
| *** jcooley_ has joined #openstack-infra | 10:24 | |
| akrivoka | hi infra team, I'm working on creating a project for tripleo ui, following the project creator guide | 10:25 |
| *** Hal1 has quit IRC | 10:26 | |
| akrivoka | according to http://docs.openstack.org/infra/manual/creators.html#update-the-gerrit-group-members, gerrit group members need to be added for the new gerrit group | 10:26 |
| akrivoka | I myself will not be core, but could somebody please add the 'tripleo-core' group to the new 'tripleo-ui-core' | 10:27 |
| akrivoka | https://review.openstack.org/#/admin/groups/1320,members | 10:27 |
| *** claudiub has joined #openstack-infra | 10:27 | |
| *** heyongli has quit IRC | 10:27 | |
| *** heyongli has joined #openstack-infra | 10:28 | |
| *** cdent has joined #openstack-infra | 10:28 | |
| *** jcooley_ has quit IRC | 10:29 | |
| *** placko_ has joined #openstack-infra | 10:29 | |
| *** claudiub|2 has quit IRC | 10:30 | |
| yolanda | jenkins03 back to normal | 10:30 |
| *** esikachev has quit IRC | 10:30 | |
| *** achanda has joined #openstack-infra | 10:31 | |
| *** achanda has quit IRC | 10:31 | |
| yolanda | jenkins05 is offline as well | 10:31 |
| *** salv-orlando has joined #openstack-infra | 10:33 | |
| yolanda | AJaeger, disk full on jenkins05 | 10:36 |
| *** _amrith_ is now known as amrith | 10:37 | |
| *** heyongli has quit IRC | 10:37 | |
| *** heyongli has joined #openstack-infra | 10:38 | |
| yolanda | infra-root, i'm seeing an increasing size of jenkins logs since 20 Mar | 10:38 |
| *** esikachev has joined #openstack-infra | 10:38 | |
| yolanda | logs are invaded with gearman debug entries,and filling up disks | 10:39 |
| *** tosky has joined #openstack-infra | 10:39 | |
| *** ihrachys has quit IRC | 10:43 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-drupal: Refactor drush deployment https://review.openstack.org/295231 | 10:47 |
| *** heyongli has quit IRC | 10:48 | |
| *** heyongli has joined #openstack-infra | 10:48 | |
| *** markmcd1 has joined #openstack-infra | 10:50 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:51 | |
| *** ihrachys has joined #openstack-infra | 10:51 | |
| *** thorst has joined #openstack-infra | 10:52 | |
| *** markmcd has quit IRC | 10:52 | |
| *** asettle has joined #openstack-infra | 10:56 | |
| openstackgerrit | amrith proposed openstack-infra/reviewstats: Added trove-dashboard to the trove project https://review.openstack.org/295723 | 10:57 |
| *** mrmartin has quit IRC | 10:57 | |
| *** heyongli has quit IRC | 10:58 | |
| *** heyongli has joined #openstack-infra | 10:58 | |
| *** thorst has quit IRC | 10:59 | |
| Nikolay_St | AJaeger: hi | 11:00 |
| *** Daisy has quit IRC | 11:00 | |
| *** Liuqing has joined #openstack-infra | 11:00 | |
| Nikolay_St | looks like we have some problems in murano kilo release after jobs rename | 11:01 |
| *** Daisy has joined #openstack-infra | 11:01 | |
| *** notnownikki has joined #openstack-infra | 11:03 | |
| *** dimtruck is now known as zz_dimtruck | 11:05 | |
| *** Daisy has quit IRC | 11:05 | |
| *** amitgandhinz has joined #openstack-infra | 11:06 | |
| *** Liuqing has quit IRC | 11:07 | |
| *** Liuqing has joined #openstack-infra | 11:08 | |
| *** heyongli has quit IRC | 11:08 | |
| *** heyongli has joined #openstack-infra | 11:08 | |
| *** Qiming has joined #openstack-infra | 11:10 | |
| AJaeger | yolanda: thanks a lot for investigation. | 11:12 |
| AJaeger | Nikolay_St: how? | 11:12 |
| *** daemontool has joined #openstack-infra | 11:13 | |
| *** gildub has quit IRC | 11:15 | |
| *** Keedya has joined #openstack-infra | 11:17 | |
| *** heyongli has quit IRC | 11:18 | |
| *** heyongli has joined #openstack-infra | 11:19 | |
| *** hamzy has quit IRC | 11:19 | |
| *** amitgandhinz has quit IRC | 11:20 | |
| *** Keedya has quit IRC | 11:20 | |
| *** jlanoux_ has joined #openstack-infra | 11:24 | |
| AJaeger | Nikolay_St: I discussed with kzaitsev_mb on #murano , should be fine again. | 11:25 |
| *** jcooley_ has joined #openstack-infra | 11:25 | |
| *** jlanoux has quit IRC | 11:26 | |
| *** jlanoux_ has quit IRC | 11:28 | |
| *** jlanoux has joined #openstack-infra | 11:29 | |
| *** heyongli has quit IRC | 11:29 | |
| *** heyongli has joined #openstack-infra | 11:29 | |
| *** Hal1 has joined #openstack-infra | 11:29 | |
| *** jcooley_ has quit IRC | 11:30 | |
| *** cdent has quit IRC | 11:31 | |
| *** smarcet has joined #openstack-infra | 11:34 | |
| *** weshay has joined #openstack-infra | 11:35 | |
| AJaeger | amrith: any more questions about the db rename? | 11:36 |
| amrith | AJaeger, yes, good morning. | 11:36 |
| amrith | I was reading your email and the comments in the review | 11:36 |
| amrith | I'm looking to speak with a couple of people on the trove team later today | 11:37 |
| amrith | and I'll update the review. I think we're good with it but I have no way of actually verifying. | 11:37 |
| *** gordc has joined #openstack-infra | 11:37 | |
| *** kushal has quit IRC | 11:37 | |
| amrith | For example, I created a patch set which depended on this change and I didn't get the expected result | 11:38 |
| AJaeger | amrith: I looked at trove/tests/util/mysql.py and concluded that you need db access | 11:38 |
| AJaeger | amrith: depends-on does not work this way with project-config. | 11:38 |
| amrith | ah, that explains that ... | 11:38 |
| AJaeger | and today you have database setup in python27 | 11:38 |
| amrith | yes we do | 11:38 |
| AJaeger | amrith: so, if you do have it today, my change gurantess that you have it also next week ;) | 11:39 |
| *** heyongli has quit IRC | 11:39 | |
| *** heyongli has joined #openstack-infra | 11:39 | |
| amrith | that sounds like a good thing ... | 11:39 |
| amrith | but there's nothing I can do to validate this? | 11:40 |
| amrith | in reviewing this, are you looking for me to concur that we need databases in our py27 job? | 11:40 |
| amrith | the answer to that is YES!!! | 11:40 |
| amrith | if that's what you are looking for, I can +2 this | 11:40 |
| AJaeger | exactly that was the answer I was looking for ;) | 11:40 |
| AJaeger | +1 only ;) | 11:40 |
| AJaeger | but yes, please do | 11:40 |
| amrith | if you were looking for something more, I'm looking for how to do this. | 11:40 |
| amrith | ok, +1 it is ... right, this is not in trove. | 11:40 |
| amrith | ok, a +1 coming up | 11:41 |
| AJaeger | no, that's all. Just confirmation that "tox -e py27" needs databases. | 11:41 |
| amrith | would this change have any additional impacts (like, performance slower?) I guess NOT because you are making the ones that don't need the db setup not have it (so those may get faster but we won't get slower). | 11:41 |
| amrith | yes? | 11:41 |
| AJaeger | we'll then rename the jobs and everything will continue to work for trove as is, just the jobs name is different | 11:41 |
| openstackgerrit | sebastian marcet proposed openstack-infra/system-config: Update enviroment id on openstackid production https://review.openstack.org/295736 | 11:41 |
| abregman | hello. can anyone review infra-ansible patches?: https://review.openstack.org/#/q/project:openstack-infra/infra-ansible+status:open+owner:%22Arie+Bregman+%253Cabregman%2540redhat.com%253E%22 | 11:42 |
| *** doug-fish has joined #openstack-infra | 11:42 | |
| *** jlanoux_ has joined #openstack-infra | 11:42 | |
| AJaeger | amrith: today python27 and python27-db is the exact same job. Next week: We take db setup away from python27, so it will be a few seconds faster | 11:42 |
| amrith | Thanks AJaeger ... +1 applied. | 11:42 |
| AJaeger | thanks, amrith | 11:43 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job https://review.openstack.org/293051 | 11:44 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Add API functionality for automatic worklist filters https://review.openstack.org/295422 | 11:44 |
| *** dprince has joined #openstack-infra | 11:45 | |
| smarcet | fungi: morning, i had a couple of changes related to new deployment of openstackid.org ( new server) | 11:45 |
| *** jlanoux has quit IRC | 11:45 | |
| openstackgerrit | Merged openstack-infra/release-tools: make the default dashboard format csv https://review.openstack.org/294776 | 11:46 |
| smarcet | fungi: https://review.openstack.org/#/c/295573/, https://review.openstack.org/#/c/295576/, https://review.openstack.org/#/c/295736/, https://review.openstack.org/#/c/295577/ | 11:46 |
| openstackgerrit | Merged openstack-infra/release-tools: sort on the managed flag before other values https://review.openstack.org/294777 | 11:46 |
| *** ihrachys has quit IRC | 11:46 | |
| *** kzaitsev_mb has quit IRC | 11:47 | |
| smarcet | fungi: i will be out this morning, but i'll be back at midday | 11:47 |
| *** smarcet has quit IRC | 11:47 | |
| *** thorst has joined #openstack-infra | 11:47 | |
| *** ldnunes has joined #openstack-infra | 11:48 | |
| *** libu has joined #openstack-infra | 11:48 | |
| *** markvoelker has joined #openstack-infra | 11:48 | |
| *** zeih has joined #openstack-infra | 11:48 | |
| *** heyongli has quit IRC | 11:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add mitaka translations for designate-dashboard https://review.openstack.org/295742 | 11:49 |
| *** doug-fis_ has joined #openstack-infra | 11:50 | |
| *** heyongli has joined #openstack-infra | 11:50 | |
| *** esikachev has quit IRC | 11:51 | |
| *** ramishra has quit IRC | 11:51 | |
| *** oanson has quit IRC | 11:51 | |
| *** ramishra has joined #openstack-infra | 11:51 | |
| *** doug-fish has quit IRC | 11:52 | |
| *** jcoufal has joined #openstack-infra | 11:52 | |
| *** zeih has quit IRC | 11:53 | |
| *** zeih has joined #openstack-infra | 11:55 | |
| *** jlanoux_ has quit IRC | 11:55 | |
| *** jlanoux has joined #openstack-infra | 11:55 | |
| AJaeger | yolanda: could you review the change above, please? | 11:56 |
| yolanda | ok | 11:56 |
| *** rhallisey has joined #openstack-infra | 11:57 | |
| *** zz_dimtruck is now known as dimtruck | 11:57 | |
| *** kgiusti has joined #openstack-infra | 11:57 | |
| *** mrmartin has joined #openstack-infra | 11:58 | |
| openstackgerrit | Merged openstack-infra/release-tools: lower case all team names to sort consistently https://review.openstack.org/294778 | 11:58 |
| openstackgerrit | Merged openstack-infra/release-tools: add PTL email to release dashboard https://review.openstack.org/294779 | 11:58 |
| kozhukalov | Guys, is there a way to get bare metal instances from nodepool for functional testing? As far as understand nodepool uses nova api to run instances. So, it should not be hard to run bare metal instances using ironic. | 11:58 |
| *** cdent has joined #openstack-infra | 11:58 | |
| sdague | yolanda: is zuul ok? The event queue is pretty big, and the number of nodes in play is pretty small | 11:59 |
| yolanda | sdague, when i checked last time, there were jenkins03 and jenkins05 down | 11:59 |
| yolanda | i restarted those, and things started to move | 11:59 |
| *** heyongli has quit IRC | 11:59 | |
| *** zeih has quit IRC | 12:00 | |
| *** heyongli has joined #openstack-infra | 12:00 | |
| yolanda | sdague all nodes that had to be created for jenkins03 and jenkins05 were deleted because they could not be attached to jenkins, so that caused overhead | 12:01 |
| sdague | yolanda: ok, cool | 12:01 |
| *** kgiusti has quit IRC | 12:01 | |
| *** esikachev has joined #openstack-infra | 12:01 | |
| *** libu has quit IRC | 12:01 | |
| *** kgiusti has joined #openstack-infra | 12:01 | |
| *** salv-orl_ has joined #openstack-infra | 12:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:03 | |
| *** salv-orlando has quit IRC | 12:05 | |
| *** ddieterly has joined #openstack-infra | 12:05 | |
| *** aysyd has joined #openstack-infra | 12:06 | |
| *** roxanaghe has joined #openstack-infra | 12:06 | |
| *** ihrachys has joined #openstack-infra | 12:06 | |
| *** Liuqing has quit IRC | 12:09 | |
| *** baoli has joined #openstack-infra | 12:10 | |
| *** heyongli has quit IRC | 12:10 | |
| *** heyongli has joined #openstack-infra | 12:10 | |
| *** Liuqing has joined #openstack-infra | 12:10 | |
| *** roxanaghe has quit IRC | 12:12 | |
| *** Liuqing has quit IRC | 12:14 | |
| openstackgerrit | Sergey Belous proposed openstack-infra/project-config: Fix $PROJECTS variable for tempest-dsvm-nova-os-vif job https://review.openstack.org/295749 | 12:15 |
| *** Liuqing has joined #openstack-infra | 12:15 | |
| sbelous | sdague: AJaeger: hi :) https://review.openstack.org/295749 please | 12:15 |
| *** amitgandhinz has joined #openstack-infra | 12:16 | |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again https://review.openstack.org/288915 | 12:17 |
| * AJaeger will be back later | 12:18 | |
| *** heyongli has quit IRC | 12:20 | |
| *** heyongli has joined #openstack-infra | 12:20 | |
| *** markvoelker has quit IRC | 12:21 | |
| *** ddieterly is now known as ddieterly[away] | 12:21 | |
| *** pradk_ has joined #openstack-infra | 12:21 | |
| *** hamzy has joined #openstack-infra | 12:21 | |
| sdague | anyone who can become a project bootstrapper help me clean up a bad tag on devstack I pushed (because I was reading the wrong docs) | 12:24 |
| abregman | is there anyone else who can review infra-ansible changes other than rcarrillocruz?: https://review.openstack.org/#/q/project:openstack-infra/infra-ansible+status:open | 12:26 |
| *** EricGonc_ has joined #openstack-infra | 12:26 | |
| *** jcooley_ has joined #openstack-infra | 12:26 | |
| *** xiangxinyong has joined #openstack-infra | 12:26 | |
| *** bdemers has quit IRC | 12:27 | |
| *** rfolco has joined #openstack-infra | 12:29 | |
| *** placko_ has quit IRC | 12:30 | |
| *** amitgandhinz has quit IRC | 12:30 | |
| *** mtanino has joined #openstack-infra | 12:30 | |
| *** EricGonc_ has quit IRC | 12:30 | |
| yolanda | sdague, i can, what do you need? | 12:30 |
| *** heyongli has quit IRC | 12:30 | |
| sdague | git push gerrit :refs/tags/stable/mitaka | 12:30 |
| *** heyongli has joined #openstack-infra | 12:30 | |
| *** pradk has quit IRC | 12:31 | |
| sdague | for devstack | 12:31 |
| *** jcooley_ has quit IRC | 12:31 | |
| sdague | I screwed up and created a tag for what should have been a branch | 12:31 |
| sdague | and I think only project bootstrappers has permissions to delete that | 12:31 |
| yolanda | ok let me do it | 12:31 |
| *** annegentle has joined #openstack-infra | 12:31 | |
| yolanda | tag deleted | 12:33 |
| *** zeih has joined #openstack-infra | 12:33 | |
| sdague | yolanda: thanks | 12:33 |
| *** pradk_ is now known as pradk | 12:34 | |
| yolanda | abregman, changes are being reviewed as part of a daily review queue. Either rcarrillocruz or myself are reviewing those on a daily basis, as time permits | 12:34 |
| yolanda | sdague no problem | 12:34 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add new tasks to os_port playbook https://review.openstack.org/295760 | 12:35 |
| abregman | yolanda: oh k. thank you. I'll wait patiently. | 12:36 |
| *** xiangxinyong456 has joined #openstack-infra | 12:37 | |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci: Adds the new '-y' required for stack-deletion when tty https://review.openstack.org/295761 | 12:38 |
| xiangxinyong456 | hi ajaeger | 12:39 |
| Zara | hm.... there are a lot of patches going round to reorder things in puppet | 12:39 |
| Zara | the storyboard one so far looks less helpful than the original; it might be worth taking a closer look at the others | 12:39 |
| yolanda | Zara yes, i'm looking at those, some of them are ok but i'm trying to -1 the ones that affect the logic | 12:39 |
| yolanda | not an easy review queue :) | 12:40 |
| Zara | yes, there are a *lot*! :) I wondered what motivated them. | 12:40 |
| *** heyongli has quit IRC | 12:40 | |
| Zara | just wanted to give people a heads up because I nearly missed it. | 12:40 |
| *** heyongli has joined #openstack-infra | 12:41 | |
| *** ddieterly[away] is now known as ddieterly | 12:41 | |
| Zara | (for the curious, it's this sort of thing: https://review.openstack.org/#/c/295372/ ) | 12:41 |
| yolanda | i don't know about the background. It has been really an incredible amount of work for Andrey Nikitin. Is ok to accomplish puppet style guide, but not if that affects the logic | 12:41 |
| *** zeih has quit IRC | 12:41 | |
| *** vdrok has quit IRC | 12:42 | |
| Zara | yeah, I looked in the style guide and didn't find anything that supported the proposed changes, though I might have missed it. | 12:42 |
| Zara | looks like I missed the wider discussion about it :/ | 12:42 |
| rcarrillocruz | i can see both sides | 12:43 |
| rcarrillocruz | logical groupings of params make sense | 12:43 |
| *** dizquierdo has quit IRC | 12:43 | |
| rcarrillocruz | yet alphabetical also are useful, for glancing params, etc | 12:43 |
| rcarrillocruz | brb, lunch | 12:43 |
| *** rlandy has joined #openstack-infra | 12:44 | |
| *** dimtruck is now known as zz_dimtruck | 12:44 | |
| EmilienM | I found something funny: https://review.openstack.org/#/c/295744/ look at gate-puppet-openstack-integration-puppet-syntax-(3|4)-dsvm-centos7 jobs, Gerrits thinks they took 0s, while if you look at the job it did not | 12:45 |
| *** rvasilets__ has joined #openstack-infra | 12:45 | |
| yolanda | about alignment ... Nested blocks must be indented by two spaces, and hash rockets within a nested block should be aligned (one space ahead of the longest attribute name). | 12:45 |
| yolanda | but that has been normally applied in arrow alignment | 12:46 |
| yolanda | i also used to space parameters, i think it improves readability, so doesn't hurt | 12:47 |
| *** markvoelker has joined #openstack-infra | 12:47 | |
| Zara | rcarrillocruz: yeah, I don't oppose alphabetical ordering, I just think that where there's a conflict, logical groupings should take precedence. then people should order things in those groups alphabetically where possible. | 12:48 |
| *** yamamoto has quit IRC | 12:48 | |
| *** xiangxi3 has joined #openstack-infra | 12:48 | |
| *** moravec has joined #openstack-infra | 12:48 | |
| *** yaume has quit IRC | 12:49 | |
| openstackgerrit | Koteswara Rao Kelam proposed openstack-infra/project-config: Adding networking-hpe project. https://review.openstack.org/295775 | 12:50 |
| *** nandhu has joined #openstack-infra | 12:50 | |
| *** naggappan has quit IRC | 12:50 | |
| rcarrillocruz | Zara: that makes sense to me | 12:51 |
| *** heyongli has quit IRC | 12:51 | |
| *** heyongli has joined #openstack-infra | 12:51 | |
| *** xiangxinyong456 has quit IRC | 12:51 | |
| yolanda | let's go with that criteria then.. i actually saw some patches trying to preserve that grouping, for example the cron parameters were together | 12:51 |
| Zara | cool | 12:51 |
| *** ddieterly has quit IRC | 12:53 | |
| pabelanger | clarkb: sure, when you have time, that works for me | 12:53 |
| openstackgerrit | Fausto Marzi proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job https://review.openstack.org/291430 | 12:56 |
| openstackgerrit | Fausto Marzi proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job https://review.openstack.org/291430 | 12:57 |
| openstackgerrit | Martin André proposed openstack/gertty: Add matchers for external references on launchpad https://review.openstack.org/295592 | 12:57 |
| *** mriedem has joined #openstack-infra | 12:59 | |
| *** gnuoy_ has joined #openstack-infra | 12:59 | |
| *** ociuhandu has quit IRC | 12:59 | |
| *** gnuoy_ has quit IRC | 12:59 | |
| pabelanger | morning | 12:59 |
| yolanda | hi | 13:00 |
| *** heyongli has quit IRC | 13:01 | |
| *** links has quit IRC | 13:01 | |
| *** heyongli has joined #openstack-infra | 13:01 | |
| EmilienM | pabelanger: I was afk yesterday evening when you pinged me about nodesets things. I'm free today (outside tripleo/puppet/...) meetings | 13:01 |
| *** dims_ has joined #openstack-infra | 13:02 | |
| *** dims has quit IRC | 13:02 | |
| *** jcooley_ has joined #openstack-infra | 13:03 | |
| pabelanger | sure, so once we land the ubuntu-trusty migration for puppet jobs, we'll need to patch the puppet modules to do the following: https://review.openstack.org/#/c/295505/ | 13:03 |
| *** rossella_s has quit IRC | 13:03 | |
| pabelanger | I believe you'll use your msync script to do it | 13:03 |
| *** jed56 has quit IRC | 13:03 | |
| pabelanger | EmilienM: so, if you agree, we can ping the infra-root and do the migration this morning | 13:04 |
| *** rossella_s has joined #openstack-infra | 13:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 13:04 |
| *** tqtran has joined #openstack-infra | 13:04 | |
| *** yaume has joined #openstack-infra | 13:04 | |
| EmilienM | pabelanger: yes, it's ok | 13:04 |
| EmilienM | we just need to add the nodesets before | 13:04 |
| EmilienM | I'm working on it | 13:04 |
| *** hashar has quit IRC | 13:06 | |
| *** |-paul-| has quit IRC | 13:06 | |
| EmilienM | pabelanger: https://review.openstack.org/295782 | 13:06 |
| EmilienM | pabelanger: if you can confirm that's ok, I can update all modules | 13:07 |
| EmilienM | degorenko: ^ will need your help on review too | 13:07 |
| *** lucasagomes is now known as lucas-hungry | 13:07 | |
| *** daemontool has quit IRC | 13:07 | |
| *** bswartz has quit IRC | 13:08 | |
| pabelanger | EmilienM: okay, so that should land after we do JJB | 13:08 |
| *** placko_ has joined #openstack-infra | 13:08 | |
| *** tqtran has quit IRC | 13:08 | |
| EmilienM | pabelanger: right but please let me first send all patches | 13:08 |
| EmilienM | so we reduce the downtime | 13:08 |
| EmilienM | I need a very few min | 13:09 |
| pabelanger | EmilienM: sure, that's fine. | 13:09 |
| *** daemontool has joined #openstack-infra | 13:09 | |
| *** placko_ has quit IRC | 13:10 | |
| *** ayoung has joined #openstack-infra | 13:11 | |
| *** heyongli has quit IRC | 13:11 | |
| *** heyongli has joined #openstack-infra | 13:12 | |
| *** bswartz has joined #openstack-infra | 13:14 | |
| *** baoli has quit IRC | 13:15 | |
| *** sshnaidm has quit IRC | 13:16 | |
| openstackgerrit | Martin André proposed openstack/gertty: Add matchers for external references on launchpad https://review.openstack.org/295592 | 13:16 |
| *** baoli has joined #openstack-infra | 13:16 | |
| *** jed56 has joined #openstack-infra | 13:17 | |
| HeOS | yolanda, regarding to https://review.openstack.org/#/c/295351/: okay, I'll save grouping of the parameters and sort them inside of a group. | 13:17 |
| *** mtanino has quit IRC | 13:17 | |
| yolanda | ++ | 13:17 |
| EmilienM | pabelanger: it's in progress: https://review.openstack.org/#/q/topic:nodesets/trusty | 13:20 |
| pabelanger | EmilienM: great | 13:20 |
| EmilienM | pabelanger: please review one of them | 13:20 |
| EmilienM | and if it's ok, you can proceed. | 13:21 |
| pabelanger | EmilienM: pretty sure they will fail however, because we don't have ubuntu-trusty landed yet | 13:21 |
| EmilienM | pabelanger: right, but just making sure the change is good | 13:21 |
| *** ddieterly has joined #openstack-infra | 13:21 | |
| pabelanger | however check experimental should work | 13:21 |
| *** heyongli has quit IRC | 13:21 | |
| *** knikolla has joined #openstack-infra | 13:22 | |
| akrivoka | I created a new repo for tripleo ui in project-config, but I forgot to include the 'upstream' clause so that the commits from another location can be imported. This patch has already been merged (https://review.openstack.org/#/c/291227/7/gerrit/projects.yaml) is there a way to fix this now? | 13:22 |
| *** _ari_ has joined #openstack-infra | 13:22 | |
| *** heyongli has joined #openstack-infra | 13:22 | |
| akrivoka | yolanda: could you perhaps help me with? ^ | 13:23 |
| akrivoka | is there a way to re-run the process? | 13:24 |
| *** sshnaidm has joined #openstack-infra | 13:24 | |
| *** Jeffrey4l has quit IRC | 13:25 | |
| *** amitgandhinz has joined #openstack-infra | 13:26 | |
| EmilienM | pabelanger: let me know when it's done, i'll recheck the patches and merge them. | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 13:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 13:27 |
| *** xiangxi3 has quit IRC | 13:28 | |
| *** xiangxinyong456 has joined #openstack-infra | 13:28 | |
| *** heyongli has quit IRC | 13:32 | |
| *** heyongli has joined #openstack-infra | 13:32 | |
| *** kgiusti has quit IRC | 13:33 | |
| *** mylu has joined #openstack-infra | 13:33 | |
| *** mylu has quit IRC | 13:34 | |
| *** erlon has joined #openstack-infra | 13:34 | |
| *** markvoelker_ has joined #openstack-infra | 13:34 | |
| *** smarcet has joined #openstack-infra | 13:34 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Fix search_users docstring https://review.openstack.org/295823 | 13:35 |
| *** tonytan4ever has joined #openstack-infra | 13:35 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: don't execute funct jobs when *.json patch https://review.openstack.org/295824 | 13:36 |
| *** markvoelker has quit IRC | 13:36 | |
| *** placko_ has joined #openstack-infra | 13:36 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Fix search_projects docstring https://review.openstack.org/295825 | 13:36 |
| *** zz_dimtruck is now known as dimtruck | 13:37 | |
| vsaienko | infra cores, please help to get this merged https://review.openstack.org/#/c/252814/ | 13:37 |
| *** markvoelker has joined #openstack-infra | 13:38 | |
| openstackgerrit | Merged openstack-infra/shade: Bug fix: Make set/unset of flavor specs work again https://review.openstack.org/294743 | 13:38 |
| openstackgerrit | Merged openstack-infra/shade: Run extra specs through TaskManager and use requests https://review.openstack.org/294928 | 13:39 |
| *** amitgandhinz has quit IRC | 13:39 | |
| pabelanger | mordred: did glean get tagged yesterday? or did we hold off on it? | 13:40 |
| *** baoli has quit IRC | 13:40 | |
| *** markvoelker_ has quit IRC | 13:40 | |
| AJaeger | xiangxinyong456: what's up? | 13:40 |
| mordred | pabelanger: I think we inadvertantly held off on it | 13:41 |
| *** eharney has joined #openstack-infra | 13:41 | |
| AJaeger | xiangxinyong456: please ask publically, maybe others can help as well | 13:41 |
| pabelanger | mordred: ack | 13:41 |
| *** baoli has joined #openstack-infra | 13:41 | |
| *** heyongli has quit IRC | 13:42 | |
| *** yamamoto has joined #openstack-infra | 13:42 | |
| *** heyongli has joined #openstack-infra | 13:42 | |
| *** yamamoto_ has joined #openstack-infra | 13:43 | |
| *** mylu has joined #openstack-infra | 13:43 | |
| pabelanger | AJaeger: do you mind helping land 287232? | 13:44 |
| *** pradk_ has joined #openstack-infra | 13:44 | |
| *** pradk has quit IRC | 13:44 | |
| *** Guest74619 is now known as BobBall | 13:44 | |
| pabelanger | AJaeger: sorry, I don't think you have +2 on it | 13:44 |
| *** pradk_ is now known as pradk | 13:44 | |
| *** xiangxi71 has joined #openstack-infra | 13:44 | |
| mordred | pabelanger: I think looking at the changelog that it's a 1.5.1 and not a 1.6.0 - you concur? | 13:44 |
| akrivoka | AJaeger: do you know if there's a way to re-run the repo creation process? I made a mistake while creating a repo for tripleo ui | 13:44 |
| AJaeger | pabelanger: indeed, no +2 for me on that one. | 13:45 |
| *** xiangxi71 has quit IRC | 13:45 | |
| AJaeger | akrivoka: best to ask clarkb, fungi, or jeblair later on whether they know a way. I can't help | 13:45 |
| akrivoka | AJaeger: will do, thank you | 13:45 |
| *** mylu has quit IRC | 13:46 | |
| *** yamamoto has quit IRC | 13:46 | |
| pabelanger | mordred: I think so | 13:47 |
| *** xiangxinyong456 has quit IRC | 13:47 | |
| *** romcheg has joined #openstack-infra | 13:48 | |
| romcheg | ci-team: Could you please re-trigger this one https://ci.fuel-infra.org/job/master.python-fuelclient.pkgs.ubuntu.review_fuel_client/244/ ? | 13:48 |
| *** tmorin has joined #openstack-infra | 13:48 | |
| *** daemontool has quit IRC | 13:48 | |
| tmorin | hi ianw, https://review.openstack.org/#/c/291103/ is now ready to progress (it was waiting for another change to merge) | 13:48 |
| tmorin | AJaeger: ^^ | 13:49 |
| pabelanger | mordred: do you mind helping with 287232? That will had functional to help clean up wiki.o.o spam | 13:49 |
| *** bdemers has joined #openstack-infra | 13:50 | |
| *** zhurong has joined #openstack-infra | 13:51 | |
| kozhukalov | guys, AJaeger , fungi, pleia2 what do you think if we deploy ironic on openstack-infra so to make it possible to launch bare metal instances using nodepool? we especially need bare metal instances for deployment tests for fuel. | 13:52 |
| *** ociuhandu has joined #openstack-infra | 13:52 | |
| *** heyongli has quit IRC | 13:52 | |
| igorbelikov | romcheg: wrong channel, please use #fuel-infra for that:) | 13:52 |
| *** heyongli has joined #openstack-infra | 13:52 | |
| AJaeger | tmorin: approved | 13:53 |
| romcheg | whoops, sorry :) | 13:53 |
| romcheg | wrong channel indeed | 13:53 |
| pabelanger | kozhukalov: we currently don't have bare metal resources to do this | 13:53 |
| tmorin | thanks AJaeger | 13:54 |
| mordred | pabelanger: done | 13:54 |
| *** daemontool has joined #openstack-infra | 13:54 | |
| pabelanger | kozhukalov: however, with zuulv3 and ansible, I see a day where bare metal provisioning is possible | 13:54 |
| mordred | yes indeed | 13:54 |
| pabelanger | Assuming there was hardware for it | 13:54 |
| *** sc68cal has joined #openstack-infra | 13:54 | |
| mriedem | AJaeger: i'm seeing several NOT_REGISTERED failures on the periodic-stable queue from this morning, on db-kilo and db-liberty jobs, is that a known issue? | 13:54 |
| *** roxanaghe has joined #openstack-infra | 13:55 | |
| AJaeger | mriedem: we renamed some jobs this morning, should be fine now. | 13:55 |
| xiangxinyong | AJaeger: ok. I find the Hacking H903 is not working. could you give some suggestions about that? | 13:55 |
| AJaeger | mriedem: that was bad timing on those jobs ,sorry | 13:56 |
| mriedem | AJaeger: ok, i'll check results of tomorrow's run then | 13:56 |
| AJaeger | xiangxinyong: can't help with that | 13:56 |
| mriedem | AJaeger: np, it's just periodic stable jobs | 13:56 |
| mordred | pabelanger: also, glean 1.5.1 released | 13:56 |
| mordred | pabelanger: (or, tag pushed at least) | 13:56 |
| mordred | greghaynes: ^^ | 13:56 |
| pabelanger | mordred: much thanks, for both | 13:56 |
| *** mylu has joined #openstack-infra | 13:57 | |
| AJaeger | mriedem: could you review https://review.openstack.org/#/c/295435/? Give it a +1 if sqlalchemy-migrate uses database when running "tox -e py27", please | 13:58 |
| kozhukalov | pabelanger: ok, what if we donate some hardware to make it possible? we are very interested in migrating our deployment tests to openstack-infra. our tests require 6 VMs running at the same time. I don't see other way to implement this currently except using bare metal instance | 13:58 |
| *** ddieterly is now known as ddieterly[away] | 13:58 | |
| *** mylu has quit IRC | 13:58 | |
| kozhukalov | we are also ready to spend our engineering resources to contribute necessary code | 13:58 |
| xiangxinyong | AJaeger: Thanks | 13:58 |
| *** roxanaghe has quit IRC | 13:59 | |
| kozhukalov | we have people who are quite experienced in using ironic | 13:59 |
| mordred | kozhukalov: it's worth a discussion ... maybe we should add sometihng to the infra meeting agenda so that we can talk about what that might look like and what the timeframe would be | 13:59 |
| *** hashar has joined #openstack-infra | 13:59 | |
| pabelanger | kozhukalov: We are currently working to bring our infracloud online, which is using community hardware. I think the current plan is to get then 100% before expanding resources. See: http://docs.openstack.org/infra/system-config/infra-cloud.html | 14:00 |
| kozhukalov | mordred: ok, i will add this topic to agenda | 14:00 |
| openstackgerrit | Merged openstack-infra/project-config: net-bgpvpn: control gate settings from rc scripts https://review.openstack.org/291103 | 14:00 |
| *** mrmartin has quit IRC | 14:00 | |
| pabelanger | I agree with mordred too, adding a topic to our weekly meeting will bring more eyes to it | 14:00 |
| kozhukalov | i just wanted to know if there are any potential obstacles except known to me | 14:00 |
| mriedem | AJaeger: done, thanks | 14:00 |
| mordred | yah - I tink there are two different obstacles - finishing zuul v3 and then having the hardware exist in a way that is non-crazy to manage | 14:01 |
| kozhukalov | probably someone tried to do this before and it was impossible for some reason, i don't know | 14:01 |
| *** mylu has joined #openstack-infra | 14:01 | |
| *** mtanino has joined #openstack-infra | 14:01 | |
| mordred | with zuul v3 we have the design for being able to deal with the thing you want/need here (we did have someone want this before, which is how it made it in to the zuul v3 design) | 14:02 |
| mordred | the hardware gets a little tricky, in that we don't want to have resources that only exist in one place | 14:02 |
| mordred | however, the HP hardware plus hardware you donate would get us to having more than one source for the resources, so I think it's a topic we can put on the table | 14:02 |
| *** heyongli has quit IRC | 14:02 | |
| *** heyongli has joined #openstack-infra | 14:03 | |
| *** rbrndt has joined #openstack-infra | 14:03 | |
| *** kushal has joined #openstack-infra | 14:03 | |
| *** asettle has quit IRC | 14:04 | |
| kozhukalov | mordred: nice, thanks for the explanation | 14:04 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI https://review.openstack.org/295844 | 14:04 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:05 | |
| *** ddieterly[away] is now known as ddieterly | 14:05 | |
| kozhukalov | mordred: if there was bare metal motivation behind zuul v3 design, could you please share a link on blueprint where i could read more details about this? | 14:05 |
| *** links has joined #openstack-infra | 14:05 | |
| mordred | yup! | 14:05 |
| fungi | also there are... security considerations. giving random proposed patches access to a physical system you can install trojan firmware and all manner of other nasty things without anyone probably even noticing | 14:06 |
| mordred | kozhukalov: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html | 14:06 |
| mordred | fungi: yah. for sure something we'd want to look at in terms of what/how a workload runs | 14:07 |
| sbelous | AJaeger: pabelanger: thanks guys, you rock! | 14:07 |
| openstackgerrit | Merged openstack-infra/puppet-mediawiki: Add SmiteSpam extension https://review.openstack.org/287232 | 14:07 |
| openstackgerrit | Merged openstack-infra/project-config: Add mitaka translations for designate-dashboard https://review.openstack.org/295742 | 14:08 |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox, name changed https://review.openstack.org/295260 | 14:08 |
| *** aysyd has quit IRC | 14:08 | |
| openstackgerrit | Merged openstack-infra/puppet-mediawiki: Set wgOpenIDUseEmailAsNickname True by default https://review.openstack.org/288785 | 14:08 |
| *** aysyd has joined #openstack-infra | 14:09 | |
| *** achanda has joined #openstack-infra | 14:09 | |
| *** lezbar has quit IRC | 14:09 | |
| *** bryan_att has joined #openstack-infra | 14:09 | |
| *** lezbar has joined #openstack-infra | 14:10 | |
| openstackgerrit | Merged openstack-infra/project-config: gerritbot/puppetopenstack: add stable/mitaka support https://review.openstack.org/295469 | 14:10 |
| *** achanda has quit IRC | 14:10 | |
| *** doug-fis_ is now known as doug-fish | 14:11 | |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox, name changed https://review.openstack.org/295260 | 14:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 14:11 |
| openstackgerrit | Merged openstack-infra/puppet-openstackid: Added apache2 service notification on ssl cert change https://review.openstack.org/295577 | 14:11 |
| mordred | SamYaple: ^^ added unit tests | 14:12 |
| *** placko_ has quit IRC | 14:12 | |
| *** baoli has quit IRC | 14:12 | |
| *** heyongli has quit IRC | 14:13 | |
| *** heyongli has joined #openstack-infra | 14:13 | |
| smarcet | fungi: morning, i already sent puppet updates for new openstackid deployment :) | 14:13 |
| *** baoli has joined #openstack-infra | 14:13 | |
| fungi | smarcet: yep, i think i've reviewed them all, approved some, will drum up second core reviews for the rest shortly | 14:14 |
| openstackgerrit | Merged openstack-infra/puppet-openstackid: Added DB seeding support https://review.openstack.org/295573 | 14:14 |
| fungi | smarcet: thanks for working on that! | 14:14 |
| smarcet | fungi: np :) | 14:16 |
| openstackgerrit | Merged openstack-infra/project-config: Fix $PROJECTS variable for tempest-dsvm-nova-os-vif job https://review.openstack.org/295749 | 14:17 |
| *** vgridnev has quit IRC | 14:19 | |
| *** vgridnev has joined #openstack-infra | 14:19 | |
| *** dims_ has quit IRC | 14:19 | |
| igorbelikov | fungi: security concerns are valid, though for fuel tests we need only non-privileged access to predeployed baremetal nodes, so pretty similar to the current scheme with nodepool images and jenkins builds | 14:19 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Don't integration test Packstack on release notes or docs patches https://review.openstack.org/295854 | 14:20 |
| *** lucas-hungry is now known as lucasagomes | 14:21 | |
| *** zhurong has quit IRC | 14:22 | |
| *** achanda has joined #openstack-infra | 14:23 | |
| *** heyongli has quit IRC | 14:23 | |
| *** heyongli has joined #openstack-infra | 14:23 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-openstackid: Order of the classes parameters is refactored https://review.openstack.org/295320 | 14:23 |
| *** esikachev has quit IRC | 14:24 | |
| *** moravec has quit IRC | 14:24 | |
| *** dimtruck is now known as zz_dimtruck | 14:24 | |
| SamYaple | mordred: ++ | 14:24 |
| openstackgerrit | Merged openstack-infra/project-config: Adds ironic neutron integration testing https://review.openstack.org/252814 | 14:25 |
| openstackgerrit | Merged openstack-infra/project-config: Move to Fedora 23 for Libvirt/LXC Nova Test https://review.openstack.org/293585 | 14:25 |
| openstackgerrit | Merged openstack-infra/puppet-accessbot: Order of the class parameters is refactored https://review.openstack.org/294959 | 14:25 |
| openstackgerrit | Merged openstack-infra/puppet-ansible: Indentation of the class parameters is refactored https://review.openstack.org/294960 | 14:25 |
| openstackgerrit | Merged openstack-infra/puppet-apps_site: Order of the class parameters is refactored https://review.openstack.org/294961 | 14:25 |
| *** moravec has joined #openstack-infra | 14:25 | |
| openstackgerrit | Deklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job https://review.openstack.org/291430 | 14:26 |
| *** esikachev has joined #openstack-infra | 14:26 | |
| *** ajmiller has joined #openstack-infra | 14:27 | |
| akrivoka | fungi: hi! I forgot to specify the 'upstream' clause while create a project for tripleo-ui: https://review.openstack.org/#/c/291227/7/gerrit/projects.yaml | 14:30 |
| akrivoka | fungi: the patch has already been merged, what can I do now to get the repo contents imported from another location? | 14:30 |
| fungi | akrivoka: yes, once a project has been created, you'll need to push in any missing changes as gerrit reviews | 14:30 |
| sdague | fungi: we're getting super early failures on multinode that are infra related | 14:31 |
| fungi | akrivoka: how many commits do you need added? | 14:31 |
| fungi | sdague: details? | 14:31 |
| akrivoka | fungi: there's no way to re-run the process? | 14:31 |
| akrivoka | fungi: more than a hundred | 14:31 |
| sdague | http://logs.openstack.org/69/295769/2/check/gate-grenade-dsvm-multinode/2eac94c//console.html#_2016-03-22_14_09_34_664 | 14:31 |
| akrivoka | fungi: 162 to be exact | 14:31 |
| fungi | akrivoka: no, a gerrit administrator would need to manually push those commits into the server, and as you can see we're pretty busy so that tends to be a very low priority activity for us | 14:31 |
| mordred | akrivoka, fungi: I've got a little time atm - what's the source repo? | 14:32 |
| akrivoka | mordred: it's here https://github.com/rdo-management/rdo-director-ui | 14:32 |
| fungi | sdague: looks like network connectivity issue between the primary node and the subnode. is that how you're interpreting it as well? | 14:33 |
| *** heyongli has quit IRC | 14:33 | |
| *** openstackgerrit has quit IRC | 14:33 | |
| *** heyongli has joined #openstack-infra | 14:33 | |
| *** openstackgerrit has joined #openstack-infra | 14:34 | |
| fungi | sdague: i wonder if this is fallout from rackspace's network maintenance. how long ago does it seem to have started getting frequent? | 14:34 |
| sdague | fungi: all week | 14:34 |
| fungi | they're apparently doing disruptive maintenance throughout this week | 14:34 |
| sdague | http://tinyurl.com/zqunqtt | 14:34 |
| fungi | sdague: that's the failure rate for multinode jobs which abort early on connectivity issues? | 14:35 |
| *** jsavak has joined #openstack-infra | 14:36 | |
| *** amitgandhinz has joined #openstack-infra | 14:36 | |
| fungi | or i guess the assumption is that grenade multinode failures in the gate pipeline are all from a common cause | 14:36 |
| *** asettle has joined #openstack-infra | 14:36 | |
| fungi | are the tempest multinode jobs exhibiting similar behavior or is this somehow grenade-only? | 14:36 |
| clarkb | e-r is tracking the specific ssh issue iirc | 14:36 |
| mordred | akrivoka: done | 14:36 |
| akrivoka | mordred: thanks so much, really appreciate it! | 14:37 |
| openstackgerrit | Merged openstack-infra/shade: Use network in neutron_available_floating_ips https://review.openstack.org/280178 | 14:37 |
| *** vgridnev has quit IRC | 14:38 | |
| fungi | would be interesting if a disproportionate number of the failures is in rackspace. no idea if this week's network maintenances (which seemed to come on _very_ short notice) are trying to fix an ongoing network issue throughout their different regions | 14:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 14:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 14:38 |
| *** jsavak has left #openstack-infra | 14:38 | |
| *** vgridnev has joined #openstack-infra | 14:40 | |
| clarkb | http://status.openstack.org/elastic-recheck/#1326813 unrelated but interesting considering no more hpcloud and all local mirrors | 14:40 |
| *** mylu has quit IRC | 14:41 | |
| *** asettle has quit IRC | 14:41 | |
| *** e0ne has quit IRC | 14:41 | |
| clarkb | fungi: sdague http://status.openstack.org/elastic-recheck/#1531187 | 14:41 |
| *** baoli has quit IRC | 14:42 | |
| sdague | this is a different exposure | 14:42 |
| *** baoli has joined #openstack-infra | 14:42 | |
| *** mylu has joined #openstack-infra | 14:42 | |
| clarkb | whats different? looks like ssh failed to subnode | 14:42 |
| *** links has quit IRC | 14:42 | |
| sdague | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ssh%3A%20connect%20to%20host%5C%22%20AND%20message%3A%5C%22Connection%20timed%20out%5C%22%20AND%20tags%3Aconsole%20AND%20voting%3A1 | 14:43 |
| *** achanda has quit IRC | 14:43 | |
| *** heyongli has quit IRC | 14:43 | |
| *** heyongli has joined #openstack-infra | 14:44 | |
| sdague | clarkb: right, it exposes a little different, as this is a hard connect fail on ssh | 14:44 |
| clarkb | ok so same issue the e-r tracking is incomplete? | 14:44 |
| sdague | yeh | 14:44 |
| sdague | or the d-g code changed and it exposes differently now | 14:45 |
| pabelanger | fungi: clarkb: I think we are in good shape to try 294203 again this morning. EmilienM has been brought up to speed on the expected changes too | 14:46 |
| *** rvasilets__ has quit IRC | 14:46 | |
| anteaya | abregman: okay great, will continue to respond to your questions on your paste | 14:46 |
| *** abregman has quit IRC | 14:46 | |
| anteaya | abregman: glad you are sticking with it | 14:47 |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: update fingerprint to match existing exposure of this issue https://review.openstack.org/295874 | 14:47 |
| EmilienM | pabelanger: I'm ready Sir | 14:47 |
| fungi | infra-core/infra-puppet-core reviewers, if you could take a look at these two quick changes i'd like to have them in before i retry to build the replacement openstackid.org server: 295736 (system-config), 295576 (puppet-openstackid) | 14:47 |
| *** dizquierdo has joined #openstack-infra | 14:49 | |
| *** zz_dimtruck is now known as dimtruck | 14:49 | |
| jeblair | Package[$php5_packages] works? | 14:49 |
| jeblair | i'm assuming $php5_packages is an array | 14:49 |
| fungi | pabelanger: on 294203 how new do images need to be to have the right fixes in place? we likely need to make sure we remove any from providers which have been failing to update longer than that | 14:49 |
| fungi | jeblair: yeah, we do that in a few other places | 14:49 |
| *** amitgandhinz has quit IRC | 14:49 | |
| jeblair | k | 14:49 |
| openstackgerrit | Merged openstack-infra/shade: Fix search_users docstring https://review.openstack.org/295823 | 14:49 |
| fungi | it's actually one of puppet's cooler features | 14:49 |
| pabelanger | fungi: ya, we should ensure images since this last friday have been online | 14:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Update enviroment id on openstackid production https://review.openstack.org/295736 | 14:50 |
| fungi | pabelanger: so saturday and later is safe but if the latest image in a provider is from friday then we need to purge it? | 14:50 |
| *** Nikolay_St has quit IRC | 14:50 | |
| pabelanger | infra-root: I need to run maintenance/update.php on wiki.o.o to load the new smitespam extension. Is there anything i need to be aware of for the database? | 14:50 |
| pabelanger | fungi: yes | 14:51 |
| jeblair | smarcet: ^ i fixed the "Depends-On" syntax in the commit message in 295736 | 14:51 |
| jeblair | fungi: sorry about your +2 ^ | 14:51 |
| smarcet | jeblair: yes sorry, was not sure about the propper syntax | 14:51 |
| clarkb | pabelanger: ya the how to upgrade wiki doc has a step on db backups iirc | 14:51 |
| fungi | jeblair: gah, i went to load it just now to +2 and gertty died on me. i have the traceback but had turned debug back off | 14:51 |
| clarkb | sdague any idea if the issue is confined to rax-ord as fungi hypothesized? | 14:52 |
| jeblair | smarcet: no problem, and there you go. :) you just use the full gerrit change id (the one that starts with "I" and then has a bunch of hex chars after it) | 14:52 |
| pabelanger | clarkb: thanks, looking now | 14:52 |
| sdague | clarkb: it's rax only, mostly ord | 14:52 |
| sdague | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ssh%3A%20connect%20to%20host%5C%22%20AND%20message%3A%5C%22Connection%20timed%20out%5C%22%20AND%20tags%3Aconsole%20AND%20voting%3A1 | 14:52 |
| jeblair | fungi: i'll take a traceback if you got it :) | 14:53 |
| *** heyongli has quit IRC | 14:54 | |
| *** heyongli has joined #openstack-infra | 14:54 | |
| jeblair | sdague, clarkb: that logstash link isn't producing results for me | 14:55 |
| sdague | ok, so what is to be done about this | 14:55 |
| sdague | jeblair: you have to set the date range | 14:55 |
| sdague | logstash always defaults to 15min | 14:55 |
| jeblair | sdague: ah that's it | 14:55 |
| *** roxanaghe has joined #openstack-infra | 14:55 | |
| *** zeih has joined #openstack-infra | 14:55 | |
| jeblair | why are we using the private ip? | 14:56 |
| clarkb | if its just rax we can get the server uuids for the affected instances, pass that along to rax and maybe stop using rax for mulyinode temporarily | 14:56 |
| sdague | jeblair: no idea, this is ansible / d-g stage of the world | 14:56 |
| clarkb | jeblair: because public doesnt work... I pushed a change yo switch it and it failed iirc | 14:57 |
| fungi | jeblair: http://paste.openstack.org/show/491460 though it's a little messy because it ended up partially overwriting the gertty display. could be a race somewhere since i tried to load that change pretty quickly after you uploaded it, probably before it was passively synced | 14:57 |
| *** tonytan4ever has quit IRC | 14:57 | |
| *** kevinsho_ has joined #openstack-infra | 14:57 | |
| *** amitgandhinz has joined #openstack-infra | 14:58 | |
| *** Swanson has quit IRC | 14:58 | |
| *** austin81 has joined #openstack-infra | 14:58 | |
| *** tonytan4ever has joined #openstack-infra | 14:58 | |
| fungi | jeblair: after restarting gertty, it doesn't crash on 295736 for me | 14:58 |
| *** sdake has joined #openstack-infra | 14:58 | |
| clarkb | https://review.openstack.org/#/c/269807/ need to get more data on that if you want to switch. in particular to cover the NAT cases | 14:58 |
| clarkb | http://logs.openstack.org/07/269807/1/check/gate-tempest-dsvm-multinode-full/457d564/ is a nat case so we do have that and it failed | 14:59 |
| openstackgerrit | Merged openstack-infra/puppet-bandersnatch: Order of the classes parameters is refactored https://review.openstack.org/294963 | 14:59 |
| openstackgerrit | Merged openstack-infra/puppet-cgit: Order of the classes parameters is refactored https://review.openstack.org/295117 | 14:59 |
| pabelanger | clarkb: thanks, database updated | 14:59 |
| openstackgerrit | Merged openstack-infra/puppet-diskimage_builder: Order of the class parameters is refactored https://review.openstack.org/295120 | 14:59 |
| *** IlyaG has joined #openstack-infra | 14:59 | |
| *** amitgandhinz has quit IRC | 15:00 | |
| fungi | clarkb: i wonder if ord has multiple backend networks and we're missing the necessary static routes to get between them? | 15:00 |
| fungi | i do recall that being the case in at least some rax regions | 15:00 |
| *** amitgandhinz has joined #openstack-infra | 15:01 | |
| *** tonytan_brb has joined #openstack-infra | 15:01 | |
| *** roxanaghe has quit IRC | 15:01 | |
| clarkb | fungi: thats possible, we get those via config drive and they should be set by glean. A few moving parts that can break there | 15:01 |
| fungi | we also recently massively increased the number of instances in ord because they wanted us to back off from iad for a bit (presumably load related) | 15:01 |
| jeblair | yeah, this is the sort of debugging that i would like to avoid by just using the public ips | 15:01 |
| jeblair | fungi: (yes) | 15:01 |
| clarkb | jeblair: unfortunately that seems to break openstack | 15:01 |
| fungi | clarkb: in that case i would posit openstack is already broken | 15:01 |
| *** angdraug has joined #openstack-infra | 15:02 | |
| clarkb | looks to be live migration related? | 15:02 |
| *** tonytan4ever has quit IRC | 15:03 | |
| *** heyongli has quit IRC | 15:04 | |
| *** heyongli has joined #openstack-infra | 15:04 | |
| *** IlyaG has quit IRC | 15:04 | |
| openstackgerrit | Austin Clark proposed openstack-infra/infra-specs: Deploy-Stackviz spec https://review.openstack.org/287373 | 15:05 |
| mordred | clarkb: doing the overlay on top of the public ips breaks openstack? that's impressive | 15:05 |
| clarkb | mordred: I dont think it is the overlay I think it is services talking to each other over NAT | 15:06 |
| clarkb | the overlay is just for nested VMs and we dont even get that far if I am reading the logs correctly | 15:06 |
| sdague | mordred: we haven't gotten to the overlay | 15:07 |
| sdague | this issue is connectivity before that, right? | 15:07 |
| clarkb | sdague: there are two things, your ssh failed over private ip and thats a really early fail. the other is when we try to use only public IPs clouds with NAT break nova things | 15:07 |
| sdague | this is the call to install openvswitch on the subnode | 15:07 |
| *** keedya has joined #openstack-infra | 15:07 | |
| sdague | clarkb: ok | 15:08 |
| sdague | is there any mitigation for this issue? | 15:08 |
| clarkb | because in theory fix broken private ip by using public ip | 15:08 |
| mordred | clarkb: yah. I mean, I continue to point out that using NAT for servers is a broken concept | 15:08 |
| clarkb | OH! I think I know why this breaks on rax | 15:08 |
| sdague | it's blocking the devstack branching, which is preventing us from actually testing upgrade to mitaka backports | 15:08 |
| clarkb | https://review.openstack.org/#/c/294871/ | 15:09 |
| clarkb | mordred ^ simple-init is broken | 15:10 |
| *** Qiming has quit IRC | 15:10 | |
| mordred | clarkb: woot! | 15:10 |
| clarkb | so ya we can get ^ in and make new images, we can bounce eth1 in the test, or just stop using rax for now | 15:10 |
| mordred | clarkb: ah. | 15:11 |
| mordred | slagle, greghaynes, SpamapS: we've got a patch we need landed in dib to unbreak multinode in rax in the gate ... any chance y'all can help? | 15:11 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci: Adds the new '-y' required for stack-deletion when tty https://review.openstack.org/295761 | 15:11 |
| mordred | (to be fair, it's a greghaynes patch - but we need help landing it since we can't do that) | 15:12 |
| SpamapS | mordred: kk | 15:12 |
| *** akaszuba has quit IRC | 15:13 | |
| SpamapS | ianw: around? https://review.openstack.org/#/c/294871 needs you ;) | 15:13 |
| HeOS | Zara, I saw, you haven't found the part about ordering of variables in puppet manifests. You can find it here: https://docs.puppetlabs.com/guides/style_guide.html#display-order-of-parameters. | 15:14 |
| clarkb | fungi you were spot on :) | 15:14 |
| *** heyongli has quit IRC | 15:14 | |
| *** heyongli has joined #openstack-infra | 15:14 | |
| SotK | HeOS: that doesn't appear to mention alphabetical ordering explicitly | 15:15 |
| *** vgridnev has quit IRC | 15:16 | |
| *** zeih has quit IRC | 15:16 | |
| *** achanda has joined #openstack-infra | 15:17 | |
| fungi | clarkb: one of my random predictions came true? that's it, i'm done for the week! | 15:17 |
| * fungi is the embodiment of the infinite monkey theorem | 15:18 | |
| *** amrith is now known as _amrith_ | 15:18 | |
| HeOS | SotK, I know. I'll don't touch group ordering where it exist. I will check ordering inside of the group. | 15:18 |
| SotK | that would work for me | 15:19 |
| dansmith | So I noticed that the nova pep8 job is failing consistently with failure to talk to git.openstack.org: https://jenkins05.openstack.org/job/gate-nova-pep8/17470/console | 15:19 |
| fungi | dansmith: i've gotten several similar failures this morning. i'm starting to think the reason rackspace wanted to do disruptive network maintenance everywhere on short notice is that something in their networks is broken (or they've broken their networks with last night's maintenance maybe) | 15:20 |
| *** harlowja_at_home has joined #openstack-infra | 15:20 | |
| dansmith | fungi: okay, seems all of a sudden this morning for me, but my six-patch set is all failing on it | 15:20 |
| fungi | i'm starting to dig in to see if i can spot any commonalities or obvious issues on the servers themselves | 15:20 |
| dansmith | so fairly consistent | 15:20 |
| AJaeger | fungi, pabelanger: ianw has in the backscroll a question about bindep-fallback and https://review.openstack.org/#/c/295521/ . Do we have some clear guideance where this belong? Should it go into one of the other elements instead? | 15:21 |
| pabelanger | AJaeger: fungi so, my thoughts on that, it should be in other-requirements for sure. But they haven't migrated there yet | 15:21 |
| fungi | AJaeger: either it goes into the bindep fallback list for now (not needed for devstack-gate based jobs?), or it goes into the posix-ish dib element | 15:22 |
| AJaeger | pabelanger: do we want to enforce other-requirements for such cases moving forward? | 15:22 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Use overcloud 'update' command instead of 'deploy' for --overcloud-update https://review.openstack.org/295896 | 15:22 |
| pabelanger | AJaeger: I'd like too, at some point. Maybe we need to talk about it in Austin | 15:22 |
| slagle | mordred: +a'd | 15:22 |
| fungi | AJaeger: pabelanger: the litmus tests is generally "would a sysadmin find it strange to not have hexdump preinstalled on a minimal server deployment?" | 15:23 |
| AJaeger | There's also one more change by ianw in this direction: https://review.openstack.org/#/c/295551/ | 15:23 |
| mordred | slagle: woot! thanks | 15:23 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Delete test stack without confirmation https://review.openstack.org/295047 | 15:23 |
| fungi | wow... http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=857&rra_id=all | 15:24 |
| *** heyongli has quit IRC | 15:24 | |
| Zara | HeOS: thanks for the link. I'm really wondering why they've used 'must' there; they seem to have cherry-picked convenient examples, when in reality there's often more tension between readability and ordering (the guide itself says to prioritize readability, but also insists that 'must' indicates a hard requirement; makes me wonder what else is unwise in that spec). | 15:25 |
| AJaeger | fungi, I have no clear answer on your litmus test for this example | 15:25 |
| *** heyongli has joined #openstack-infra | 15:25 | |
| AJaeger | Sorry, have to go offline again | 15:25 |
| fungi | AJaeger: i've needed to install hexdump on servers myself in the past, so it seems more appropriate for the bindep-fallback.txt | 15:25 |
| fungi | but it's open to interpretation | 15:25 |
| fungi | it's not immediately apparent to me where all the memory usage on git.o.o is tied up | 15:26 |
| pabelanger | AJaeger: fungi: Right, but I think now that we have other-requirements.txt in place, it makes it easy for projects to add the dependency moving forward. I think, if we need to add hexdump to some large number of project (50%) or something, we could then move it up into a dib element? | 15:27 |
| fungi | pabelanger: no, unless it's really surprising to have missing from a minimal posix-ish environment, i think it belongs in the fallback list (and eventually in individual projects' lists) | 15:28 |
| fungi | because installing something is easier than uninstalling it if you need it to _not_ be present for some reason | 15:28 |
| daemontool | hi, can I get +A here please https://review.openstack.org/#/c/291430/ | 15:28 |
| *** vgridnev has joined #openstack-infra | 15:28 | |
| HeOS | Zara, I suppose, we can use the following schemas: 1. if we have splitting of parameters, we can follow the rules inside of the group; 2. if we have no splitting, the we can sort all the parameters alphabetically. | 15:28 |
| fungi | pabelanger: and also it provides clearer documentation as to what packages are really required | 15:28 |
| daemontool | ^^ Modify gate-freezer-api-devstack-dsvm job | 15:29 |
| *** okay22many has quit IRC | 15:29 | |
| jeblair | fungi: that is strange. that graph doesn't seem to agree with 'free'. | 15:31 |
| *** stupidnic has joined #openstack-infra | 15:31 | |
| *** pc_m has quit IRC | 15:31 | |
| fungi | i think snmp and free are in disagreement as to what counts as "used" memory. `free` says most of the available memory is consumed by buffers/cache (which makes sense) but the graph is showing something completely different | 15:31 |
| fungi | yeah | 15:31 |
| pabelanger | fungi: okay, I can get behind that | 15:32 |
| HeOS | Guys, should I add 'recheck' comment here: https://review.openstack.org/#/c/295130/, because this request has V-2 from Jenkins. I see, that job was aborted by timeout in the log. | 15:32 |
| jeblair | fungi: i wonder if something about snmp is different on that server (rhel?) | 15:32 |
| HeOS | *Jenkins? | 15:32 |
| fungi | pabelanger: yeah, i'm trying to treat other-requirements.txt (and by extension bindep-fallback.txt for now) as documentation in a convenient format that devs can easily do something like `tox -e bindep` and find out what packages they're missing which should be installed to be able to run tests | 15:33 |
| fungi | jeblair: oh, that seems plausible | 15:33 |
| fungi | jeblair: the oid mapping in the system mib could be different somehow i guess | 15:33 |
| stupidnic | Not sure if this is the right place or not, but I am unable to set my Contact Information on my gerrit account in review.openstack.org | 15:33 |
| fungi | though i thought that was provided directly by net-snmp | 15:33 |
| fungi | (not that it would be the first time rh changed something like that) | 15:34 |
| vsaienko | infra please review https://review.openstack.org/#/c/294128/ | 15:34 |
| openstackgerrit | Merged openstack-infra/openstackid: Reduce noise on error log https://review.openstack.org/294907 | 15:34 |
| openstackgerrit | Merged openstack-infra/puppet-elastic_recheck: Order of the classes parameters is refactored https://review.openstack.org/295132 | 15:34 |
| *** bdemers has quit IRC | 15:34 | |
| fungi | stupidnic: were you following http://docs.openstack.org/infra/manual/developers.html#account-setup or using some instructions you found somewhere else? | 15:34 |
| stupidnic | fungi: yep that is what I was following | 15:34 |
| igorbelikov | HeOS: timeout on zuul cloner, yeah, use recheck | 15:34 |
| openstackgerrit | Merged openstack-infra/puppet-elasticsearch: Order of the class parameters is refactored https://review.openstack.org/295134 | 15:35 |
| *** heyongli has quit IRC | 15:35 | |
| HeOS | igorbelikov, thanks. | 15:35 |
| fungi | stupidnic: you already created your foundation member account, and you set your primary e-mail address there to the same thing as your preferred e-mail address in gerrit? | 15:35 |
| *** heyongli has joined #openstack-infra | 15:35 | |
| *** MarkAtwood has joined #openstack-infra | 15:35 | |
| stupidnic | Yep. I am only having an issue saving my actual Contact Information in the settings of my gerritt account | 15:35 |
| stupidnic | I accepted the agreement, but I couldn't fill in the information there either | 15:36 |
| fungi | stupidnic: right, the reason i ask is that the "file contact information" step in gerrit is actually a proxy mechanism to hit the foundation member system api and link up your e-mail address to a member account | 15:36 |
| stupidnic | fungi: Ah I see. It is the same information | 15:36 |
| *** bdemers has joined #openstack-infra | 15:37 | |
| stupidnic | Let me make 100% sure | 15:37 |
| fungi | stupidnic: when you joined the openstack foundation you did a foundation member application, not a community member account? | 15:37 |
| stupidnic | I did a community | 15:37 |
| fungi | we tried to be very specific about that in the instructions since people seem to miss that all the time | 15:37 |
| stupidnic | Guilty as charged then | 15:37 |
| fungi | "Your member level needs to be Foundation Member (not Community Member)." | 15:37 |
| *** pc_m has joined #openstack-infra | 15:38 | |
| dansmith | clarkb: any progress on the mirror thing for the ceph job? | 15:38 |
| *** yamahata has quit IRC | 15:38 | |
| stupidnic | Maybe a little <strong> on that might help | 15:38 |
| pabelanger | okay, so the new SmiteSpam extension on wiki.o.o is working well. I'm in the processes of cleaning up and blocking new accounts from the spam attack last month. It will also make it easy for wiki admins to quickly stop spammers too | 15:38 |
| *** ddieterly is now known as ddieterly[away] | 15:38 | |
| clarkb | dansmith: mordred left comments I need to address | 15:38 |
| dansmith | clarkb: okay | 15:39 |
| stupidnic | fungi: so can I change the account type on the foundation side? | 15:39 |
| *** EricGonczer_ has joined #openstack-infra | 15:39 | |
| dansmith | clarkb: mordred: we've already let in one regression that the job would have caught, so we're kinda itching to get it back on :D | 15:39 |
| clarkb | and apparently I need to make a new afsvolume | 15:39 |
| dansmith | clarkb: which is not a thing anyone says in 2016 | 15:39 |
| fungi | stupidnic: yes, there's a button on your profile, in the top-right corner i think | 15:39 |
| pabelanger | so far, 70 accounts blocked, over 280 pages deleted and climbing | 15:39 |
| fungi | stupidnic: if you have trouble finding it, i think there's a fairly accurate screenshot of it at https://ask.openstack.org/question/56720 | 15:40 |
| *** Liuqing has quit IRC | 15:40 | |
| pabelanger | https://wiki.openstack.org/wiki/Special:Log/block | 15:40 |
| fungi | stupidnic: should say something like "make me a foundation member" | 15:40 |
| stupidnic | fungi: yep that's spot on the problem I was having... thank you and sorry for the trouble | 15:40 |
| *** eil397 has joined #openstack-infra | 15:40 | |
| Zara | HeOS: I'm not sure what you mean by splitting parameters. I'm not opposed to minimal reordering, eg: http://paste.openstack.org/show/491469/ . but putting things in alphabetical order, or moving all things without default values to the top makes it harder for developers to find what code does what. and once you group by purpose (or say, where a file ends up in a system), there tends not to be much ro | 15:41 |
| fungi | stupidnic: no apologies needed, this is way more complicated than it should have been. i blame lawyers ;) | 15:41 |
| Zara | om left for alphabetical ordering | 15:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 15:41 |
| *** eil397 has quit IRC | 15:41 | |
| *** ddieterly[away] is now known as ddieterly | 15:41 | |
| *** eil397 has joined #openstack-infra | 15:41 | |
| stupidnic | fungi: I think that can be applied to a lot of things these days... contact info updated... coolio, thanks again. | 15:42 |
| fungi | stupidnic: great--happy hacking! | 15:42 |
| *** stupidnic has left #openstack-infra | 15:42 | |
| Zara | HeOS: so I'm questioning why following this set of requirements is seen as desirable; they don't seem to hold up to real-world examples. | 15:42 |
| openstackgerrit | Merged openstack-infra/puppet-etherpad_lite: Order of the classes parameters is refactored https://review.openstack.org/295138 | 15:42 |
| fungi | jeblair: i'm digging around and finding lots of reports which back up your assertion that those oids map to different counters on rhel 7. still looking for something definitive though | 15:43 |
| HeOS | I mean, we can have set of parameters for cron, another set for gerrit. So, we can sort them all alphabetically and we have a mix of them as result. Or we can split them on "groups" in global list of parameters and sort it alphabetically inside of them and add a special comment before them. | 15:43 |
| openstackgerrit | Murad Awawdeh proposed openstack-infra/project-config: new project: horizon-mellanox https://review.openstack.org/295260 | 15:43 |
| openstackgerrit | Merged openstack-infra/puppet-exim: Order of the class parameters is refactored https://review.openstack.org/295139 | 15:44 |
| openstackgerrit | Merged openstack-infra/puppet-gerritbot: Order of the class parameters is refactored https://review.openstack.org/295140 | 15:44 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Fix the path to get_host_info.sh https://review.openstack.org/295913 | 15:44 |
| jeblair | fungi: cool, i've started poking at snmp queries, will resume after breakfast | 15:44 |
| *** arif-ali has quit IRC | 15:44 | |
| *** heyongli has quit IRC | 15:45 | |
| *** heyongli has joined #openstack-infra | 15:45 | |
| HeOS | Zara, according to your link: we can sort alphabetically "Installation parameters" group, "storyboard.conf parameters" group and so on. | 15:45 |
| *** dims has joined #openstack-infra | 15:45 | |
| Zara | HeOS: but if you do, you no longer sort them by where the files end up on the system | 15:46 |
| Zara | HeOS: and to me, that is more important | 15:46 |
| Zara | for someone debugging, etc | 15:46 |
| fungi | jeblair: their manual indicates UCD-SNMP-MIB::memory should have similar values to the free command https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/sect-System_Monitoring_Tools-Net-SNMP.html#sect-System_Monitoring_Tools-Net-SNMP-Retrieving (§18.7.4.2) | 15:47 |
| openstackgerrit | Merged openstack-infra/puppet-github: Order of the class parameters is refactored https://review.openstack.org/295142 | 15:48 |
| openstackgerrit | Merged openstack-infra/puppet-graphite: Order of the class parameters is refactored https://review.openstack.org/295144 | 15:48 |
| *** Sukhdev has joined #openstack-infra | 15:48 | |
| openstackgerrit | Merged openstack-infra/puppet-hound: Order of the class parameters is refactored https://review.openstack.org/295146 | 15:48 |
| *** jlanoux has quit IRC | 15:49 | |
| openstackgerrit | Merged openstack-infra/puppet-httpd: Order of the classes parameters is refactored https://review.openstack.org/295157 | 15:49 |
| openstackgerrit | Merged openstack-infra/puppet-iptables: Indentation of the class parameters is refactored https://review.openstack.org/295164 | 15:49 |
| openstackgerrit | Merged openstack-infra/system-config: Update enviroment id on openstackid production https://review.openstack.org/295736 | 15:49 |
| HeOS | Zara, as I understand, those variables are presented here as examples, because production-related values are stored at hiera. Is it right? | 15:49 |
| anteaya | did any infra-root read in backscroll that yolanda is finding the new jenkinsii are filling up logging much faster than the old jenkinsii? | 15:49 |
| *** arif-ali has joined #openstack-infra | 15:50 | |
| *** bpokorny has joined #openstack-infra | 15:50 | |
| pabelanger | fungi: so, looking at nodepool image-list, it looks like we might need to check ovh-bhs1, rax-dfw, and vexxhost-ca-ymq-1 for ubuntu-trusty images | 15:50 |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Order of the classes parameters is refactored https://review.openstack.org/294962 | 15:50 |
| anteaya | http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-03-22.log.html#t2016-03-22T10:36:51 | 15:50 |
| *** woodster_ has joined #openstack-infra | 15:51 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add mirror server https://review.openstack.org/238414 | 15:51 |
| *** tonytan_brb is now known as tonytan4ever | 15:53 | |
| *** mylu has quit IRC | 15:53 | |
| openstackgerrit | Deklan Dieterly proposed openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job https://review.openstack.org/291430 | 15:53 |
| fungi | anteaya: yep, saw that. i wonder if we turned on extra debug logging in the new gearman plugin build | 15:53 |
| fungi | zaro: ^ | 15:53 |
| anteaya | fungi: thanks | 15:53 |
| *** mylu has joined #openstack-infra | 15:53 | |
| fungi | pabelanger: that change is only going to be impacted by stale ubuntu-trusty i take it? | 15:54 |
| anteaya | thought it might be worth a prod to avoid starting to lose jenkinsii | 15:54 |
| fungi | pabelanger: i know ovh-bhs1 has been consistently failing to get uploads | 15:54 |
| *** vgridnev has quit IRC | 15:54 | |
| pabelanger | fungi: Ya, the order images will be missing cron | 15:54 |
| *** vgridnev has joined #openstack-infra | 15:55 | |
| *** dims_ has joined #openstack-infra | 15:55 | |
| *** vgridnev has quit IRC | 15:55 | |
| *** dims has quit IRC | 15:55 | |
| fungi | pabelanger: i guess we'll need to determine whether the temporary loss of ubuntu-trusty capacity in those locations outweighs being able to move forward with moving the puppet jobs | 15:55 |
| clarkb | fungi yesterday ovh, rax, and vexxhost all had upload problems | 15:55 |
| Zara | HeOS: some defaults are used, some aren't, and in general I'd be wary of assuming downstreams will always use something other than the examples with minimal tweaks. | 15:55 |
| *** heyongli has quit IRC | 15:55 | |
| Zara | I believe we only use hiera for private data | 15:55 |
| *** heyongli has joined #openstack-infra | 15:55 | |
| *** roxanagh_ has joined #openstack-infra | 15:56 | |
| *** sridhar_ram has joined #openstack-infra | 15:56 | |
| vincentll | I would be glad if someone could have a look at our submission for integration into OpenStack CI: https://review.openstack.org/#/c/283157/ | 15:58 |
| *** sridhar_ram has quit IRC | 15:59 | |
| *** sridhar_ram has joined #openstack-infra | 15:59 | |
| *** roxanagh_ has quit IRC | 16:01 | |
| *** tmorin has left #openstack-infra | 16:01 | |
| Zara | (sotk has linked me https://github.com/openstack-infra/system-config/blob/master/manifests/site.pp#L594 , so you can see where there's overlap) | 16:01 |
| *** yamamoto_ has quit IRC | 16:02 | |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Use the 'zuul' qualifier in all loggers https://review.openstack.org/295925 | 16:02 |
| * SotK apologises that that isn't a git.openstack.org link | 16:02 | |
| anteaya | SotK: thanks | 16:02 |
| johnsom | Hi, I setup a new launchpad for neutron-lbaas-dashboard, but commits closing bugs are not updating the bug in launchpad. What step did I miss? | 16:02 |
| *** IlyaG has joined #openstack-infra | 16:02 | |
| anteaya | SotK: you can link line numbers in git.openstack.org too | 16:02 |
| anteaya | SotK: but right now cgit doesn't have the ability to highlight ranges of line numbers | 16:03 |
| openstackgerrit | Merged openstack-infra/puppet-drupal: Order of the classes parameters is refactored https://review.openstack.org/295130 | 16:03 |
| *** tphummel has joined #openstack-infra | 16:03 | |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Support GitHub PR webhooks https://review.openstack.org/163117 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Merge pull requests from github reporter https://review.openstack.org/243250 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Fix job hierarchy bug. https://review.openstack.org/192457 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Encapsulate determining the event purpose https://review.openstack.org/247487 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: support github pull reqeust labels https://review.openstack.org/247421 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Better merge message for GitHub pull reqeusts https://review.openstack.org/280667 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Add 'push' and 'tag' github webhook events. https://review.openstack.org/191207 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Add 'pr-comment' github webhook event https://review.openstack.org/239203 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Support for dependent pipelines with github https://review.openstack.org/247500 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Configurable SSH access to GitHub https://review.openstack.org/239138 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: GitHub file matching support https://review.openstack.org/292376 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Allow github trigger to match on branches/refs https://review.openstack.org/258448 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Log GitHub API rate limit https://review.openstack.org/292377 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Allow list values in template parameters. https://review.openstack.org/191208 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Add basic Github Zuul Reporter. https://review.openstack.org/191312 | 16:04 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Support for github commit status https://review.openstack.org/239303 | 16:04 |
| *** tqtran has joined #openstack-infra | 16:04 | |
| anteaya | johnsom: did you follow all the steps from: http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad | 16:04 |
| yolanda | fungi, i cleaned logs on jenkins03 and jenkins05. Both had 100% disk space used, and jenkins stopped | 16:04 |
| johnsom | anteaya Yes, that is the guide I followed. | 16:04 |
| fungi | yolanda: oh, i missed you saying the disks had filled up entirely, just saw you say logs were a lot bigger now | 16:04 |
| *** keedya has quit IRC | 16:05 | |
| JayF | SamYaple: (Back from vacation) Those IPA images are published here: http://tarballs.openstack.org/ironic-python-agent/coreos/ (ipa-coreos.tar.gz) | 16:05 |
| anteaya | johnsom: did you follow all the steps? | 16:05 |
| JayF | Hey, if someone could cleanup http://tarballs.openstack.org/ironic-python-agent/coreos/files/, it'd probably be a good idea -- that was an old location for the kernel/initrd and anything there is like, 1.5+y old | 16:05 |
| *** heyongli has quit IRC | 16:05 | |
| anteaya | johnsom: in gerrit/projects.yaml in project-config did you add a group? | 16:05 |
| *** heyongli has joined #openstack-infra | 16:06 | |
| anteaya | johnsom: did you set the owner of the team in launchpad? Set the owner of the team to ‘openstack-admins’ | 16:06 |
| johnsom | anteaya The group is neutron in projects.yaml, the maintainers group does include openstack-admins. | 16:08 |
| *** tonytan4ever has quit IRC | 16:08 | |
| openstackgerrit | Merged openstack/diskimage-builder: Remove all interface configs for simple-init https://review.openstack.org/294871 | 16:08 |
| *** tonytan4ever has joined #openstack-infra | 16:09 | |
| *** apoorvad has joined #openstack-infra | 16:09 | |
| anteaya | johnsom: can you share the url to the launchpad page? | 16:09 |
| jeblair | fungi: git.o.o has both "MemFree" and "MemAvailable" in /proc/meminfo; our other hosts only have "MemFree" | 16:09 |
| johnsom | anteaya https://launchpad.net/neutron-lbaas-dashboard | 16:09 |
| jeblair | fungi: i don't know what the difference is yet, but the number for MemAvailable looks like the one we want from git.o.o. | 16:10 |
| jeblair | fungi: so i think that's at least the explanation for the difference. | 16:10 |
| anteaya | fungi yolanda jenkins01 and 02 have almost no used space, jenkins 03 and 05 were the only ones to fill up, jenkins 04, 06 and 07 are about half full | 16:10 |
| jeblair | fungi: (our cacti uses MemFree) | 16:10 |
| fungi | jeblair: i have a very vague recollection of having to use one for debian and openbsd servers, and the other for red hat servers with mrtg years ago | 16:10 |
| jeblair | https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773 | 16:11 |
| *** kzaitsev_mb has quit IRC | 16:11 | |
| *** rkukura has quit IRC | 16:11 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: update fingerprint to match existing exposure of this issue https://review.openstack.org/295874 | 16:11 |
| fungi | jeblair: ahh, yeah that makes sense. free is sort of glossing over it now (compare to what newer top versions display) | 16:12 |
| *** annegentle has quit IRC | 16:12 | |
| *** yamahata has joined #openstack-infra | 16:12 | |
| jeblair | fungi: having said that, i'm not sure that the meaning of MemFree has changed.... | 16:13 |
| anteaya | johnsom: look at the neutron launchpad group: https://bugs.launchpad.net/neutron on the right you see the bug supervisor is "Neutron Bugs" | 16:13 |
| zaro | morning | 16:13 |
| anteaya | morning zaro | 16:13 |
| anteaya | johnsom: the owner of "Neutron Bugs" is "OpenStack Administrators" https://launchpad.net/~neutron-bugs | 16:14 |
| openstackgerrit | Merged openstack-infra/reviewstats: Added trove-dashboard to the trove project https://review.openstack.org/295723 | 16:14 |
| SamYaple | JayF: cool. yea i looked it over. looks good. docker-engine does support build proxy now though | 16:14 |
| anteaya | johnsom: you need to create a group that is your bug supervisor and the owner of the group you create has to be OpenStack Administrators | 16:14 |
| johnsom | anteaya Hmmm, Octavia works without that setting | 16:14 |
| jeblair | fungi: so either we changed kernel versions in late november and the meaning of MemFree changed with it, or starting in late november, something changed on the server causing MemFree to continually decrease (without corresponding increases in other metrics?) | 16:15 |
| JayF | SamYaple: we should probably use the same versino you all do then to get that feature | 16:15 |
| anteaya | johnsom: I dont' understand all the intracacies of launchpad | 16:15 |
| JayF | SamYaple: was thinking about it, and we should probably add some method to devstack that just sets up docker, then both of our gates use it | 16:15 |
| anteaya | johnsom: but that is my advice | 16:15 |
| clarkb | we upgraded to centos7 around then | 16:15 |
| johnsom | anteaya Ok, thank you for your help. I will give that a shot | 16:15 |
| zaro | anteaya, fungi : nothing was changed in regards to gearman plugin logging. that might be more jenkins core logging | 16:15 |
| anteaya | johnsom: you are welcome | 16:15 |
| JayF | SamYaple: as I told jroll, if we align our gate with your gate, less likely for a broken devstack to sit around waiting for us to discover it :P | 16:15 |
| SamYaple | JayF: we dont devstack, so im not familiar with the devstack things | 16:15 |
| JayF | oh. | 16:16 |
| *** heyongli has quit IRC | 16:16 | |
| *** mylu has quit IRC | 16:16 | |
| anteaya | zaro: okay, yolanda mentioned lots of gearman debug entries | 16:16 |
| *** heyongli has joined #openstack-infra | 16:16 | |
| zaro | anteaya, fungi : this change to reduce logging was proposed log ago but never approved: https://review.openstack.org/#/c/82627/ | 16:16 |
| fungi | jeblair: clarkb: yep, the switch from centos 6 to 7 would explain that timing | 16:16 |
| *** EricGonczer_ has quit IRC | 16:17 | |
| anteaya | zaro: can you freshen that patch up? | 16:17 |
| *** EricGonczer_ has joined #openstack-infra | 16:17 | |
| anteaya | zaro: you have a workflow -1 on it waiting on an abandoned patch | 16:17 |
| jeblair | fungi, clarkb: well *that* kernel patch didn't change the meaning of memfree; maybe some other one did, or maybe something else was going on... | 16:18 |
| *** jaosorior has quit IRC | 16:18 | |
| smarcet | fungi: i will be using a new tool to do load testing on openstackid.org | 16:19 |
| smarcet | fungi: https://loader.io | 16:19 |
| *** mylu has joined #openstack-infra | 16:19 | |
| *** hashar has quit IRC | 16:19 | |
| smarcet | fungi: its need a domain verification | 16:19 |
| zaro | anteaya, fungi : i take that back, this was the change that added more logging, https://review.openstack.org/#/c/180371/ | 16:19 |
| *** mylu has quit IRC | 16:19 | |
| *** mylu has joined #openstack-infra | 16:19 | |
| smarcet | fungi: need to drop a file on the site root (production) | 16:19 |
| fungi | smarcet: thanks for the heads up. do they validate by e-mail to the administrative contact for the domain registration? | 16:19 |
| smarcet | fungi: in order to perform verification | 16:20 |
| fungi | can have them send it to hostmaster@openstack.org and i can approve via callback link or whatever | 16:20 |
| smarcet | fungi: u should upload this file to server root | 16:20 |
| smarcet | fungi: http://openstackid.org/loaderio-1b3d65aa06f6c888c8d2f21f4a0169e1.txt | 16:20 |
| zaro | anteaya, fungi : yeah, i can refreshen. | 16:20 |
| *** jlanoux has joined #openstack-infra | 16:21 | |
| smarcet | fungi: sorry , this https://loader.io/targets/1b3d65aa06f6c888c8d2f21f4a0169e1/verification_file | 16:21 |
| *** cdent has quit IRC | 16:21 | |
| fungi | smarcet: should we add that via puppet, or is it a very temporary authorization? | 16:21 |
| *** e0ne has joined #openstack-infra | 16:21 | |
| anteaya | zaro: I think that was created to add more logging for jjb updates, as the reason jjb updates fail sometimes were not getting logged | 16:21 |
| smarcet | fungi: its kind of temp thing | 16:21 |
| anteaya | zaro: thank you | 16:21 |
| *** yamamoto has joined #openstack-infra | 16:21 | |
| *** bswartz has quit IRC | 16:21 | |
| anteaya | zaro: so obviously we need to find a happy medium | 16:21 |
| smarcet | fungi : i could sent u out the instructions by email if u agree | 16:21 |
| fungi | smarcet: might make more sense to puppet the file in since we'll need to do it on both the old and new servers probably, but i can do it by hand if we're not planning to reuse that service after this | 16:22 |
| anteaya | I'm curious why the amount of disk space is so different on the jenkinsii | 16:22 |
| anteaya | sorry disk space used | 16:22 |
| smarcet | fungi: or i could add it to git repo | 16:22 |
| fungi | smarcet: i can go ahead and add it manually. just a sec | 16:23 |
| smarcet | fungi: ok i will sent out the instructions by email :) | 16:23 |
| fungi | smarcet: yeah, i can't get to the loader.io url you mentioned without an account | 16:24 |
| fungi | so will need a copy of the file contents | 16:25 |
| *** sdake has quit IRC | 16:25 | |
| smarcet | fungi: just sent it out | 16:25 |
| smarcet | let me know if u need something else | 16:25 |
| smarcet | :) | 16:25 |
| smarcet | and thx u! | 16:25 |
| *** heyongli has quit IRC | 16:26 | |
| *** mylu has quit IRC | 16:26 | |
| *** Swami has joined #openstack-infra | 16:26 | |
| *** heyongli has joined #openstack-infra | 16:26 | |
| daemontool | anyone can help review https://review.openstack.org/#/c/255349/ it's for the python-freezerclient repo | 16:26 |
| *** sdake has joined #openstack-infra | 16:29 | |
| daemontool | and https://review.openstack.org/#/c/291430/ dsvm gate job change to support tempest plugin | 16:29 |
| dims_ | anteaya : AJaeger : please peek at this when you get a chance - https://review.openstack.org/#/c/295531/ - needed add py34 functional testing | 16:29 |
| *** sc68cal has quit IRC | 16:29 | |
| anteaya | hey dims_ | 16:30 |
| dims_ | hi anteaya | 16:30 |
| anteaya | I'll take a look | 16:30 |
| *** Jeffrey4l has joined #openstack-infra | 16:30 | |
| *** kgiusti has joined #openstack-infra | 16:31 | |
| anteaya | dims_: AJaeger has a comment on the patch already | 16:31 |
| fungi | smarcet: i've created a /srv/openstackid/w/public/loaderio-1b3d65aa06f6c888c8d2f21f4a0169e1.txt file containing the string loaderio-1b3d65aa06f6c888c8d2f21f4a0169e1 and it's visible at http://openstackid.org/loaderio-1b3d65aa06f6c888c8d2f21f4a0169e1.txt now | 16:31 |
| *** sridhar_ram has left #openstack-infra | 16:32 | |
| *** mylu has joined #openstack-infra | 16:32 | |
| fungi | smarcet: let me know when you're ready to have that deleted again | 16:32 |
| smarcet | fungi: done | 16:32 |
| smarcet | tnx a lot | 16:32 |
| fungi | smarcet: does the file need to remain there for subsequent tests, or can it be removed already? | 16:32 |
| smarcet | fungi: u could remove it | 16:32 |
| smarcet | tnx | 16:32 |
| fungi | smarcet: done. thanks! | 16:33 |
| smarcet | thx ! | 16:33 |
| jeblair | fungi: that returns " | 16:33 |
| jeblair | Whoops, looks like something went wrong. | 16:33 |
| jeblair | for me | 16:34 |
| fungi | jeblair: yep, i already deleted it | 16:34 |
| *** skolekonov is now known as _skolekonov | 16:34 | |
| jeblair | ah | 16:34 |
| fungi | apparently just needed to be there ephemerally for the service to confirm we controlled that site | 16:34 |
| fungi | i guess it assumes we control it in perpetuity now ;) | 16:34 |
| *** mikelk has quit IRC | 16:35 | |
| *** esikachev has quit IRC | 16:35 | |
| fungi | i'm sure it's just a safeguard to prevent their service from being used to launch a denial of service attack | 16:35 |
| *** EricGonczer_ has quit IRC | 16:35 | |
| *** Nikolay_St has joined #openstack-infra | 16:35 | |
| *** mylu has quit IRC | 16:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 16:36 |
| *** _amrith_ is now known as amrith | 16:36 | |
| *** yamamoto has quit IRC | 16:36 | |
| *** heyongli has quit IRC | 16:36 | |
| *** heyongli has joined #openstack-infra | 16:36 | |
| *** matrohon has quit IRC | 16:37 | |
| *** catherineD has joined #openstack-infra | 16:37 | |
| openstackgerrit | Fausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client https://review.openstack.org/255349 | 16:37 |
| *** smarcet has quit IRC | 16:37 | |
| *** earlephilhower has joined #openstack-infra | 16:39 | |
| *** sc68cal has joined #openstack-infra | 16:40 | |
| openstackgerrit | Merged openstack-infra/shade: Fix search_projects docstring https://review.openstack.org/295825 | 16:41 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release https://review.openstack.org/293155 | 16:43 |
| clarkb | SamYaple: mordred ^ I think that should be ready now except for the need to add a new afs volume. How do I determine the size of the ceph repo? | 16:43 |
| pabelanger | clarkb: if we holding off on the puppet migration, do you want to cover what is needed for the new centos wheel server? | 16:44 |
| clarkb | pabelanger: not necessarily hodling off on puppet migration just trying to catch up on everything else from yesterday | 16:44 |
| *** catherineD has left #openstack-infra | 16:44 | |
| clarkb | pabelanger: I also have to pop out this afternoon so not a huge help if puppet migration has trouble | 16:44 |
| clarkb | pabelanger: are we holding off due to images? | 16:45 |
| pabelanger | clarkb: hold off not the right work, but ya, need to ensure we have ubuntu-trusty images in 3 clouds | 16:45 |
| *** akshai has joined #openstack-infra | 16:45 | |
| mordred | clarkb: I determined the size of the ubuntu trusty repo by building one locally with reprepro and then using du | 16:46 |
| clarkb | pabelanger: gotcha, in that case yes lets make a centos7 mirror slave | 16:46 |
| *** eil397 has quit IRC | 16:46 | |
| pabelanger | Also, I assume if we did a new glean release today, we'd need to restart nodepool-builder to pick it up? | 16:46 |
| *** tosky has quit IRC | 16:46 | |
| pabelanger | clarkb: WFM | 16:46 |
| mordred | pabelanger: noh - shouldn't need to | 16:46 |
| mordred | pabelanger: dib should get the latest glean when it builds the images | 16:46 |
| greghaynes | pabelanger: nope - that should get picked up without a restart | 16:46 |
| *** heyongli has quit IRC | 16:46 | |
| *** dims_ has quit IRC | 16:46 | |
| pabelanger | mordred: greghaynes: great | 16:47 |
| *** heyongli has joined #openstack-infra | 16:47 | |
| *** eil397 has joined #openstack-infra | 16:47 | |
| *** tosky has joined #openstack-infra | 16:47 | |
| clarkb | pabelanger: step one for launching a long lived server is log into puppetmaster and make a system-config clone in your homedir | 16:47 |
| clarkb | pabelanger: then make a virtualenv with shade and python-novaclient==2.31.0 installed | 16:47 |
| *** cdent has joined #openstack-infra | 16:48 | |
| clarkb | pabelanger: then you can run the shade-launch-node script in system-config except it doesn't exist anymore | 16:48 |
| jeblair | clarkb: does the global install of shade not work? | 16:48 |
| clarkb | jeblair: the global install of novaclient doesn't work | 16:48 |
| clarkb | jeblair: fungi discovered this yesterday | 16:48 |
| pabelanger | ianw: So, I think we can approve 294363 now | 16:48 |
| jeblair | ah | 16:48 |
| pabelanger | clarkb: okay, let me do that now | 16:48 |
| jeblair | clarkb: why the specific novaclient version in your venv instruction? | 16:49 |
| fungi | clarkb: jeblair: well, the global install of novaclient _does_ work with the launch script from the master branch tip of system-config | 16:49 |
| clarkb | jeblair: because that version is old enough to have the novaclient v1_1 api | 16:49 |
| clarkb | fungi: oh! so for pabelanger a venv won't be required | 16:49 |
| clarkb | since this isn't a duplicate server its a brand new one | 16:49 |
| fungi | clarkb: right, i was having to roll back to a pre-ansible version because of the uuid as hostname oversight | 16:50 |
| clarkb | pabelanger: in that case ignore the venv you should be fine | 16:50 |
| pabelanger | clarkb: okay | 16:50 |
| jeblair | what's the uuid as hostname fix? | 16:50 |
| *** MCoLo__ has joined #openstack-infra | 16:50 | |
| clarkb | jeblair: use old launch script which is why you need a venv | 16:50 |
| fungi | launching a new server with the current launch-node.py probably works (i hear it's been tested). launching a replacement with the same name as an existing server does not yet | 16:50 |
| jeblair | clarkb: that sounds like a workaround, not a fix :) | 16:51 |
| clarkb | my suggestion yesterday was to revert until it is working, but I don't think that is urgent if mordred or someone is able to work on a real fix | 16:51 |
| clarkb | jeblair: yup :) | 16:51 |
| *** vgridnev has joined #openstack-infra | 16:51 | |
| fungi | ansible decides servers with the same name should be identified by their uuids instead, and then proceeds to configure the system under the assumption that its hostname is actually that uuid | 16:51 |
| *** vincentll has quit IRC | 16:51 | |
| jeblair | yeah, trying to find out who's working on that fix and what it will look like | 16:51 |
| *** tqtran has quit IRC | 16:52 | |
| *** pcaruana has quit IRC | 16:52 | |
| *** akshai has quit IRC | 16:52 | |
| yolanda | clarkb, fungi, jeblair, the ansible fix to allow server replacement with shade-launch-node merged last weeks. Is there any other problem? | 16:52 |
| clarkb | pabelanger: so anyways, once you have that cloned you can look at the readme in the launch dir and it has basic directions of how to run the launch script | 16:52 |
| fungi | which also results in system-specific puppet not being applied because the hostname doesn't match anything in the global site manifest | 16:52 |
| *** MCoLo has quit IRC | 16:52 | |
| *** akshai has joined #openstack-infra | 16:52 | |
| pabelanger | clarkb: right | 16:52 |
| anteaya | so for folks looking to be involved in the infra cloud call that crinkle has set up, an email has gone out to folks with a pdf that shows some layout, call happens in just over an hour | 16:52 |
| anteaya | layout is kind of detailed | 16:52 |
| fungi | yolanda: i was trying with master branch tip and got the behavior i just described | 16:53 |
| openstackgerrit | Khai Do proposed openstack-infra/gearman-plugin: less verbose logging https://review.openstack.org/82627 | 16:53 |
| clarkb | pabelanger: I prefer to run that in screen because it can take some time | 16:53 |
| pabelanger | anteaya: thanks for the reminder | 16:53 |
| zaro | fungi, anteaya : ^ | 16:53 |
| anteaya | pabelanger: welcome | 16:53 |
| anteaya | zaro: thank you | 16:53 |
| clarkb | pabelanger: then when that script is done it prints out commands to update dns which you can do if the server apperas to have built properly | 16:53 |
| fungi | yolanda: was teh fix in system-config or somewhere else? | 16:53 |
| yolanda | in ansible devel branch, mordred approved that | 16:53 |
| fungi | yolanda: or are you thinking of clarkb's patch to ansible? that's something else entirely | 16:54 |
| pabelanger | clarkb: run the script from the launch folder or top level? | 16:54 |
| fungi | different issue | 16:54 |
| pabelanger | or does it matter | 16:54 |
| clarkb | pabelanger: from the launch folder | 16:54 |
| clarkb | at least thats where I always run it | 16:54 |
| yolanda | ah i was thinking on clarkb's fix yes | 16:54 |
| pabelanger | ok | 16:54 |
| clarkb | pabelanger: one thing I would do is check the flavor used by the other slave | 16:54 |
| *** akshai_ has joined #openstack-infra | 16:54 | |
| clarkb | pabelanger: and use that flavor, also make sure you use a pvhvm image for centos as they seem to perform a bit better | 16:54 |
| fungi | yolanda: so in this case it's not that ansible is picking the _wrong_ uuid (that's what clarkb fixed) but that it's deciding the replacement server's name _is_ the uuid | 16:54 |
| clarkb | the default flavor and image are probably not what we want for these slaves | 16:55 |
| yolanda | when i used shade-launch-node for gerrit, i got puppet running there. I wonder if that was a consequence of the ansible bug | 16:56 |
| pabelanger | clarkb: okay, first question. Do I need to manually add myself to the puppet group it looks like? Surprised puppet manifests did do it | 16:56 |
| clarkb | pabelanger: I thought that was puppeted, if it isn't puppeted then yes | 16:56 |
| pabelanger | okay, will get a patch up too | 16:57 |
| *** heyongli has quit IRC | 16:57 | |
| *** heyongli has joined #openstack-infra | 16:57 | |
| *** keedya has joined #openstack-infra | 16:57 | |
| *** akshai has quit IRC | 16:58 | |
| jeblair | clarkb, yolanda, mordred: i created a bug: https://storyboard.openstack.org/#!/story/2000520 | 16:58 |
| *** kzaitsev_mb has joined #openstack-infra | 16:58 | |
| jeblair | because i haven't heard that anyone is working on fixing that right now | 16:59 |
| pabelanger | clarkb: appear to be using 4GB Performance for ubuntu trusty wheel | 16:59 |
| clarkb | pabelanger: that sounds about right and I would just use that for the centos mirror builder | 16:59 |
| anteaya | I thought clarkb had a patch up to ansbile on github about that | 16:59 |
| clarkb | anteaya: separate problems, my patch got merged | 17:00 |
| fungi | jeblair: yeah, i started to look into it but the interactions with the ansible openstack inventory script began to get me twisted around | 17:00 |
| jeblair | that's the second time someone thought clarkb's patch addressed that problem...so.... clarkb do you have a link to that on github? | 17:00 |
| fungi | hoping someone with deeper anisble grokkage can determine whether this is something which is trivial to fix or will need a significant redesign | 17:00 |
| *** kevinsho_ has quit IRC | 17:00 | |
| anteaya | clarkb: ah sorry | 17:00 |
| anteaya | clarkb: and yay, merged patch | 17:01 |
| clarkb | jeblair: I can dig it up, it will not fix this problem as far as I know | 17:01 |
| anteaya | jeblair: https://github.com/ansible/ansible/pull/14984 | 17:01 |
| clarkb | anteaya: thanks | 17:01 |
| anteaya | welcome | 17:01 |
| *** flepied1 has quit IRC | 17:01 | |
| jeblair | ok. i have added a comment to the bug indicating that patch is unrelated :) | 17:01 |
| anteaya | jeblair: thanks | 17:01 |
| anteaya | sorry for being confused | 17:02 |
| *** lucasagomes is now known as lucas-afk | 17:02 | |
| anteaya | I should just have that on a t-shirt | 17:02 |
| jeblair | anteaya: no problem, you were not the only one | 17:02 |
| anteaya | thanks | 17:02 |
| *** Guest91251 has quit IRC | 17:02 | |
| *** dims has joined #openstack-infra | 17:02 | |
| *** ddieterly is now known as ddieterly[away] | 17:03 | |
| fungi | the only thing they have in common afaik is that they involve the uuid-using behavior of asnible openstack inventory when faced with multiple servers of the same name. it's a shallow similarity at best | 17:03 |
| *** rossella_s has quit IRC | 17:03 | |
| *** rossella_s has joined #openstack-infra | 17:03 | |
| anteaya | I'm all about shallow | 17:04 |
| *** flepied has joined #openstack-infra | 17:04 | |
| anteaya | kind of a hallmark really | 17:04 |
| fungi | that one clarkb fixed was clearly a logic bug. the current issue seems more like a design oversight when thinking we could rely on the ansible inventory while building replacement servers (i'm hoping i'm wrong, because that sounds like a much deeper issue with the whole model) | 17:04 |
| mordred | nope | 17:04 |
| mordred | it' sa very simple bug | 17:04 |
| mordred | hostname: name="{{ inventory_hostname.split('.', 1)[0] }}" | 17:05 |
| mordred | that should not be inventory_hostname in this case | 17:05 |
| mordred | in the set_hostnames playbook | 17:05 |
| * mordred will make a fix | 17:05 | |
| pabelanger | clarkb: looks like our server name is wheel-mirror-centos-7-amd64.slave.openstack.org | 17:05 |
| anteaya | yay a fix! | 17:05 |
| fungi | mordred: to the rescue! | 17:05 |
| fungi | mordred: i'm more than happy to test that, as i have a current need to deploy a replacement server | 17:06 |
| *** ddieterly[away] is now known as ddieterly | 17:06 | |
| clarkb | pabelanger: wheel-mirror-ubuntu-trusty-amd64.slave.openstack.org is a thing so your string looks good to me | 17:06 |
| mordred | woot! | 17:06 |
| pabelanger | clarkb: okay, running launch.py now | 17:06 |
| pabelanger | chmod 755 launch-node.py first :) | 17:07 |
| clarkb | pabelanger: if it runs successfully what I like to do is login and make sure puppet looks right, got the correct image etc. Then if that looks fine add to dns using the commands launch prints out. | 17:07 |
| yolanda | mordred , nice ... so then my shade-launch-node running puppet on gerrit, was actually running due to the bug on ansible with incorrect settings on inventory | 17:07 |
| *** heyongli has quit IRC | 17:07 | |
| yolanda | thanks to the bug | 17:07 |
| pabelanger | clarkb: ack | 17:07 |
| *** heyongli has joined #openstack-infra | 17:07 | |
| fungi | mordred: would this also explain why when there are duplicate servers we were getting reports submitted into puppetdb with the uuid instead of a hostname? | 17:08 |
| *** regXboi has joined #openstack-infra | 17:08 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Give 0775 to launch-node.py https://review.openstack.org/295959 | 17:09 |
| pedroalvarez | Hi infra! this system-config patch has been hanging around for 6 days and even has a +2, could anyone have a look at it please? please? :) https://review.openstack.org/#/c/293367/ | 17:09 |
| anteaya | hey pedroalvarez | 17:10 |
| anteaya | pedroalvarez: we are a tad busy at present | 17:10 |
| *** ddieterly is now known as ddieterly[away] | 17:10 | |
| *** baoli has quit IRC | 17:10 | |
| mordred | fungi: nope, that's a different issue that I believe nibz was working on | 17:10 |
| pedroalvarez | anteaya: hello! :) | 17:10 |
| anteaya | pedroalvarez: it is always nice to see you :) | 17:10 |
| mordred | yolanda, fungi: ACTUALLY - the bug in launch-node is in a different place. but it's the same general thing | 17:10 |
| pedroalvarez | anteaya: right, I understand, I never know when people is busy here or just talking about things :) too much traffic always | 17:11 |
| *** baoli has joined #openstack-infra | 17:11 | |
| mordred | we're actually writing out a temporary inventory that uses the uuid as the hostname :) | 17:11 |
| anteaya | we too are keen to see storyboard able to send email | 17:11 |
| pedroalvarez | good to know :) | 17:11 |
| *** sputnik13 has joined #openstack-infra | 17:11 | |
| anteaya | pedroalvarez: always, it just needs one more system-config core reviewer, I'll try to see if I can find a good time to remind folks in the next day or so | 17:12 |
| anteaya | pedroalvarez: thanks for your patience :) | 17:12 |
| *** zeih has joined #openstack-infra | 17:12 | |
| pedroalvarez | anteaya: as always, thanks for your help :) | 17:12 |
| anteaya | pedroalvarez: my pleasure :) | 17:12 |
| *** ddieterly[away] is now known as ddieterly | 17:13 | |
| *** dims has quit IRC | 17:13 | |
| anteaya | yay, thanks jeblair! | 17:13 |
| *** nadya_ has quit IRC | 17:13 | |
| anteaya | and fungi | 17:14 |
| pedroalvarez | indeed, thanks!! :) | 17:14 |
| fungi | jeblair beat me to it, but change looked fine | 17:14 |
| anteaya | thank you for looking as well | 17:14 |
| pabelanger | clarkb: everything worked as expected, but I get the following warnings after the dns output: http://paste.openstack.org/show/491481/ | 17:14 |
| anteaya | hard to match jeblair for speed :) | 17:14 |
| *** scheuran has quit IRC | 17:15 | |
| *** rkukura has joined #openstack-infra | 17:15 | |
| pabelanger | I did not use sudo to run launch-node | 17:15 |
| *** degorenko is now known as _degorenko|afk | 17:15 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Don't set the host's name as UUID https://review.openstack.org/295962 | 17:15 |
| mordred | yolanda, fungi: ^^ | 17:15 |
| clarkb | pabelanger: I think thats ok, its trying to make the ansible inventory up to date but it should self correct that the next time around | 17:15 |
| clarkb | pabelanger: I wouldnt worry about that if everything ese is working | 17:15 |
| *** sdake_ has joined #openstack-infra | 17:16 | |
| openstackgerrit | Merged openstack-infra/shade: Create clouds in Functional Test base class https://review.openstack.org/295296 | 17:16 |
| openstackgerrit | Merged openstack-infra/shade: Deal with is_public and ephemeral in normalize_flavors https://review.openstack.org/295387 | 17:16 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Don't set the host's name as UUID https://review.openstack.org/295962 | 17:16 |
| *** sdake has quit IRC | 17:16 | |
| mordred | sorry, left off the story id last time | 17:16 |
| *** ddieterly is now known as ddieterly[away] | 17:17 | |
| *** dprince has quit IRC | 17:17 | |
| *** heyongli has quit IRC | 17:17 | |
| pabelanger | clarkb: server looks good 4GB and puppet appears happy | 17:17 |
| *** heyongli has joined #openstack-infra | 17:17 | |
| pabelanger | clarkb: setting up dns now | 17:18 |
| pedroalvarez | oh, it's the its-storyboard plugin actually running in gerrit? | 17:18 |
| clarkb | pabelanger: great. Once dns is reoslving the last step is to add it to the jenkins master | 17:18 |
| mordred | pedroalvarez: nope. I just figured adding the story footer would be friendly | 17:18 |
| *** baoli has quit IRC | 17:18 | |
| *** yamamoto has joined #openstack-infra | 17:18 | |
| pedroalvarez | mordred: ah, right. I wondered for a second if the comment in the story pointing to the patch was done automatically, but nah | 17:19 |
| pabelanger | clarkb: Hmm, ERROR: You must provide a username via either --os-username or env[OS_USERNAME] from rack-dns | 17:19 |
| yolanda | mordred, if we don't use server-id in ansible but name, how does ansible knows which entry is the right one? | 17:19 |
| pabelanger | guess it is not using shade? | 17:19 |
| clarkb | pabelanger: did you source the files it says to source? | 17:19 |
| yolanda | target is enough? | 17:19 |
| clarkb | pabelanger: it should be like: source file foo run these two commands, now source file bar and run these other two commands | 17:19 |
| mordred | yolanda: it's not using the openstack inventory | 17:19 |
| mordred | yolanda: it's only using the static file we write out | 17:19 |
| mordred | yolanda: so there is no right one or wrong one in this case | 17:20 |
| yolanda | ah ok | 17:20 |
| *** bswartz has joined #openstack-infra | 17:20 | |
| mordred | yolanda: I mena, I think we were being extra careful, which in this case is a good thing :) | 17:20 |
| pabelanger | clarkb: only thing it says is to activate venv: . ~root/rackdns-venv/bin/activate | 17:20 |
| *** rkukura has quit IRC | 17:20 | |
| clarkb | pabelanger: ya that should setup your usernames and all that, maybe another file permissions issue? | 17:20 |
| clarkb | pabelanger: oh! | 17:21 |
| clarkb | I wonder if with the ansible shade setup you didn't source the other rax file first | 17:21 |
| clarkb | and maybe that needs to pull some stuff in | 17:21 |
| *** baoli has joined #openstack-infra | 17:21 | |
| pabelanger | Oh, I know why | 17:22 |
| pabelanger | maybe | 17:22 |
| clarkb | pabelanger: if its not perms do . ~root/ci-launch/openstackci-rs-nova.sh then . ~root/rackdns-venv/bin/activate possibly | 17:22 |
| pabelanger | let me try something | 17:22 |
| clarkb | pabelanger: ok | 17:22 |
| mordred | yah. you'd be missing ~root/ci-launch/openstackci-rs-nova.sh from the shade bits | 17:22 |
| *** daemontool has quit IRC | 17:22 | |
| pabelanger | okay, if that, the instructions need updating | 17:23 |
| pabelanger | http://paste.openstack.org/show/491485/ | 17:23 |
| pabelanger | it is later in the steps | 17:23 |
| *** flepied has quit IRC | 17:23 | |
| clarkb | pabelanger: ya so before shade we always had that sourced prior beacuse it was required but now that shade is used os-c-c clouds.yaml is used over env file | 17:23 |
| clarkb | pabelanger: so ya we would just need to update the directions that are printed or make the dns env file self sufficient | 17:24 |
| *** tqtran has joined #openstack-infra | 17:24 | |
| *** amrith is now known as _amrith_ | 17:24 | |
| pabelanger | that helped, but I'm getting 401: {u'message': u'Unauthorized', u'code': 401, u'details': u'Cannot manage PTR records for devices on a different account.'} | 17:24 |
| *** nadya has joined #openstack-infra | 17:24 | |
| openstackgerrit | Merged openstack-infra/system-config: Add sender email for storyboard.openstack.org https://review.openstack.org/293367 | 17:25 |
| anteaya | pedroalvarez: ^^ | 17:25 |
| pedroalvarez | anteaya: we are really happy right now :) | 17:25 |
| Zara | :D :D :D | 17:26 |
| anteaya | yay! | 17:26 |
| clarkb | pabelanger: note the difference between openstack-rs-nova.sh and openstackci-rs-nova.sh | 17:26 |
| pabelanger | clarkb: Ah | 17:26 |
| pabelanger | thanks | 17:26 |
| anteaya | Zara: :) | 17:26 |
| clarkb | pabelanger: I would start wit ha clean env, source openstackci-rs-nova.sh first, run the dns commands as printed | 17:26 |
| pabelanger | clarkb: okay, that worked | 17:27 |
| *** heyongli has quit IRC | 17:27 | |
| *** krtaylor has quit IRC | 17:27 | |
| pabelanger | clarkb: and for AAAA records, I need to use the openstack-rs-nova.sh right? | 17:28 |
| *** heyongli has joined #openstack-infra | 17:28 | |
| clarkb | pabelanger: the first pair of dns commands make reverse pointers, the secound pair are A and AAAA | 17:28 |
| clarkb | pabelanger: and just follow the directions as printed | 17:28 |
| *** mriedem1 has joined #openstack-infra | 17:28 | |
| pabelanger | Hmm | 17:29 |
| pabelanger | "details": "Limit of 500 records per domain has been reached.", | 17:29 |
| pabelanger | for A record | 17:29 |
| pabelanger | which is last command I need to run | 17:29 |
| *** zz_ja is now known as zz_zz_ja | 17:30 | |
| pabelanger | clarkb: ^ | 17:30 |
| clarkb | nice! | 17:30 |
| fungi | pabelanger: yep, need to use their webui | 17:30 |
| clarkb | wow | 17:30 |
| fungi | their hacked up dns client utility is terrible for anything other than setting stuff | 17:30 |
| fungi | their webui is terrible too (paginated scrolling, arbitrary order, not searchable) | 17:30 |
| fungi | but at least you can, with some effort, get to the entries you need | 17:31 |
| *** mriedem has quit IRC | 17:31 | |
| rcarrillocruz | heh | 17:31 |
| *** Kiall has quit IRC | 17:31 | |
| rcarrillocruz | that reminds me the webui of hp public cloud for DNS | 17:31 |
| rcarrillocruz | :/ | 17:31 |
| rcarrillocruz | laggish | 17:31 |
| *** Sukhdev has quit IRC | 17:31 | |
| rcarrillocruz | unsearchable | 17:31 |
| fungi | i think it was designed with the expectation that almost nobody has more than 10 rrs in a domain | 17:31 |
| rcarrillocruz | ++ | 17:31 |
| *** mriedem has joined #openstack-infra | 17:31 | |
| *** earlephilhower has quit IRC | 17:32 | |
| *** mriedem has quit IRC | 17:32 | |
| *** yamamoto has quit IRC | 17:32 | |
| *** mriedem has joined #openstack-infra | 17:33 | |
| *** Kiall has joined #openstack-infra | 17:33 | |
| *** mriedem1 has quit IRC | 17:33 | |
| *** lucas-afk is now known as lucasagomes | 17:34 | |
| anteaya | clarkb: this is your patch to add mitaka stable branch jobs, it is missing a job definition: https://review.openstack.org/#/c/295499/3 | 17:34 |
| anteaya | clarkb: do you want to fix or do you want me to fix? | 17:35 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Fix MAGNUM_IMAGE_NAME setting https://review.openstack.org/295969 | 17:35 |
| pabelanger | fungi: so, I guess I need to delete a subdomain then, if they have capped us at 500 for openstack.org | 17:35 |
| *** rkukura has joined #openstack-infra | 17:35 | |
| clarkb | anteaya: why don't you comment and I can fix. It will need rebasing on top of AJaeger's unittest db work | 17:35 |
| anteaya | I have commented with the error from config layout | 17:36 |
| mordred | fungi: might we return to the idea of maybe not using rax dns anymore? I know that involves a bryce conversation though | 17:36 |
| pabelanger | is pypi.slave.openstack.org still a thing? Or no because we are on AFS | 17:36 |
| pabelanger | maybe not | 17:36 |
| pabelanger | that will be for jenkins | 17:36 |
| *** ifarkas has quit IRC | 17:36 | |
| fungi | pabelanger: oh, you got the 500 limit error when trying to add a record? hrm, that's definitely new | 17:36 |
| pabelanger | fungi: Yup | 17:36 |
| pabelanger | I cannot add anything more | 17:37 |
| mordred | pypi.mirror.openstack.org is not a thing | 17:37 |
| fungi | and yeah, pypi.slave.openstack.org is no more. it was replaced by release.slave.openstack.org | 17:37 |
| mordred | oh, I mean pypi.slave | 17:37 |
| mordred | so, that'll get us past this one thing | 17:37 |
| pabelanger | safe to delete then? | 17:37 |
| fungi | yep | 17:37 |
| mordred | yup | 17:37 |
| mordred | Im guessing they're counting A records, AAAA records and PTR records all as things? | 17:37 |
| *** heyongli has quit IRC | 17:38 | |
| fungi | probably not ptr records since those go in other zones | 17:38 |
| mordred | oh, right | 17:38 |
| *** heyongli has joined #openstack-infra | 17:38 | |
| pabelanger | \o/ dns added now | 17:38 |
| fungi | but a, aaaa, cname (and also a handful of srv, ns, maybe the soa) | 17:38 |
| mordred | woot | 17:38 |
| mordred | yah | 17:38 |
| mordred | fungi: I, for one, would be happy for us to run some dns and do dns sec and stuff | 17:39 |
| pabelanger | clarkb: okay, so next step is add it to jenkins.o.o as a static slave? | 17:39 |
| clarkb | pabelanger: yup, for that just copy how the other one is configured | 17:39 |
| clarkb | pabelanger: and that should be it. After that its just making jobs run there | 17:39 |
| *** angdraug has quit IRC | 17:40 | |
| fungi | mordred: agreed. i already do that for my personal domains (debian instances in rax-iad and rax-syd running bind 10 with dnssec properly implemented) | 17:40 |
| mordred | fungi: as do jeblair and I for ours | 17:41 |
| fungi | mordred: also, that gets you dns servers with ipv6 addresses (which rackspace's dns service crazily does not have) | 17:41 |
| *** krtaylor has joined #openstack-infra | 17:41 | |
| mordred | fungi: bonghits | 17:41 |
| mordred | fungi: and we could do geodns stuff | 17:41 |
| *** daemontool has joined #openstack-infra | 17:42 | |
| Zara | btw, someone has asked if there are any updates on wiki openid login issues, here: https://storyboard.openstack.org/#!/story/2000513 ; I don't know the current status but anyone with an update may wish to respond. | 17:42 |
| * Zara is excited that she may not need to be a notifications daemon for much longer | 17:42 | |
| anteaya | Zara: you are a great notifications daemon | 17:43 |
| anteaya | Zara: but I do understand if you want to explore other roles | 17:43 |
| pabelanger | clarkb: okay added into jenkins. Waiting for jenkins to connect via SSH | 17:43 |
| *** krotscheck has quit IRC | 17:43 | |
| Zara | anteaya: I'm a notifications daemon that runs 5 hours late; there are probably better options out there. | 17:44 |
| anteaya | Zara: anytime is the right time | 17:44 |
| anteaya | Zara: but I do take your point | 17:44 |
| *** pcaruana has joined #openstack-infra | 17:44 | |
| Zara | :) | 17:44 |
| *** tonytan4ever has quit IRC | 17:45 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: ssh less https://review.openstack.org/295973 | 17:45 |
| *** derekh has quit IRC | 17:45 | |
| *** jlanoux has quit IRC | 17:45 | |
| pabelanger | clarkb: okay, so centos-7 wheel mirror is missing the jenkins user. Which makes me think something did provision correctly | 17:45 |
| pabelanger | checking logs now | 17:45 |
| clarkb | pabelanger: if manifests/site.pp is too specific for the trusty builder it may not have puppeted correctly | 17:46 |
| clarkb | pabelanger: you could fix that and rerun puppet | 17:46 |
| *** yaume has quit IRC | 17:47 | |
| *** akrivoka has quit IRC | 17:48 | |
| pabelanger | clarkb: looks like it should have matched. So, what steps are needed to hook the new server into ansible? An inventory update? | 17:48 |
| *** heyongli has quit IRC | 17:48 | |
| pabelanger | I guess launch-node handled that already | 17:48 |
| clarkb | pabelanger: that and getting the root user on puppetmsater to accept the ssh host key for that server | 17:48 |
| *** heyongli has joined #openstack-infra | 17:48 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Pretty print json output when writing to a file https://review.openstack.org/290840 | 17:48 |
| pabelanger | ok, let me look at ci.openstack.org docs | 17:49 |
| *** tonytan4ever has joined #openstack-infra | 17:49 | |
| *** jordanP has quit IRC | 17:49 | |
| *** lezbar has quit IRC | 17:51 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 17:52 |
| mordred | yolanda: ^^ this is not my finest hour of coding | 17:53 |
| mordred | yolanda: please do not look at the differences between that patch and the one before it | 17:53 |
| *** tonytan4ever has quit IRC | 17:53 | |
| *** tonytan4ever has joined #openstack-infra | 17:53 | |
| * yolanda will do now :) | 17:53 | |
| *** nadya has quit IRC | 17:54 | |
| yolanda | we all missed that | 17:54 |
| *** lezbar__ has joined #openstack-infra | 17:54 | |
| *** doublek has joined #openstack-infra | 17:54 | |
| *** flepied has joined #openstack-infra | 17:55 | |
| fungi | crinkle: great points on 295576 | 17:56 |
| *** zeih has quit IRC | 17:56 | |
| fungi | i should have read it more closely | 17:56 |
| crinkle | fungi: i didn't -1 because it already had 2 +2 and wouldn't break the world if it went in | 17:56 |
| pabelanger | okay, won't be attending infracloud call. Going to finish up this wheel mirror | 17:57 |
| fungi | crinkle: well, i am a little worried about it reporting on ever run and running php4enmod when not needed | 17:57 |
| *** lucasagomes is now known as lucas-dinner | 17:57 | |
| pabelanger | 2016-03-22 17:56:23,504 p=29729 u=root | wheel-mirror-centos-7-amd64.slave.openstack.org : ok=0 changed=0 unreachable=1 failed=0 | 17:57 |
| fungi | er, on every | 17:57 |
| pabelanger | should get it this time around | 17:57 |
| *** tosky has quit IRC | 17:57 | |
| *** dprince has joined #openstack-infra | 17:57 | |
| clarkb | pabelanger: unreachable may be due to the ssh key, just accept it as root | 17:57 |
| pabelanger | clarkb: yup, did that too | 17:57 |
| crinkle | fungi: yeah i do not like that very much | 17:58 |
| clarkb | pabelanger: great | 17:58 |
| *** d0ugal has quit IRC | 17:58 | |
| *** angdraug has joined #openstack-infra | 17:58 | |
| fungi | time to dial into the conference call | 17:58 |
| *** heyongli has quit IRC | 17:58 | |
| pabelanger | good luck | 17:58 |
| *** heyongli has joined #openstack-infra | 17:59 | |
| *** d0ugal has joined #openstack-infra | 17:59 | |
| *** baoli has quit IRC | 17:59 | |
| *** baoli has joined #openstack-infra | 18:00 | |
| fungi | pabelanger: AJaeger: on the hexdump change, they wanted it available for devstack so adding to bindep-fallback.txt isn't an option | 18:00 |
| *** tosky has joined #openstack-infra | 18:01 | |
| *** angdraug has quit IRC | 18:01 | |
| HeOS | yolanda, if you have no objections more, could you replace you w-1 by any CR score? https://review.openstack.org/#/c/294964/ | 18:01 |
| yolanda | oh i didn't want w-1 but cr -1 ... | 18:02 |
| *** salv-orlando has joined #openstack-infra | 18:02 | |
| *** pvaneck has joined #openstack-infra | 18:02 | |
| *** smarcet has joined #openstack-infra | 18:02 | |
| HeOS | :) | 18:02 |
| * yolanda needs to leave for a while | 18:03 | |
| *** EricGonczer_ has joined #openstack-infra | 18:03 | |
| HeOS | yolanda, thank you. :) | 18:03 |
| smarcet | fungi: seems that openstackid.org is down | 18:04 |
| *** dimtruck is now known as zz_dimtruck | 18:04 | |
| fungi | smarcet: are you load testing right now? | 18:04 |
| *** sambetts is now known as sambetts|afk | 18:04 | |
| smarcet | nope | 18:04 |
| fungi | i'm on a conference call, but i'll try to have a look | 18:04 |
| smarcet | fungi: ok thx u | 18:04 |
| *** zz_dimtruck is now known as dimtruck | 18:04 | |
| *** jsavak has joined #openstack-infra | 18:04 | |
| fungi | load's low | 18:04 |
| smarcet | it think that is something related to laravel | 18:04 |
| pabelanger | fungi: smarcet I can check | 18:04 |
| smarcet | bc the error message | 18:04 |
| smarcet | just need the log file | 18:05 |
| smarcet | on /app/storage/logs | 18:05 |
| smarcet | to figure it out | 18:05 |
| smarcet | weird thing is that dev is up | 18:05 |
| smarcet | https://openstackid-dev.openstack.org/ | 18:05 |
| eil397 | down for me also | 18:05 |
| smarcet | latest changes were done at pupet level | 18:05 |
| *** salv-orl_ has quit IRC | 18:05 | |
| *** annegentle has joined #openstack-infra | 18:05 | |
| pabelanger | okay, on the system. Where am I looking for logs | 18:06 |
| fungi | smarcet: /srv/openstackid/storage/logs/log-apache2handler-2016-03-22.txt or apache logs? | 18:06 |
| *** bpokorny has quit IRC | 18:06 | |
| smarcet | fungi: /srv/openstackid/storage/logs/log-apache2handler-2016-03-22.txt o | 18:06 |
| rcarrillocruz | fungi: you just doing audio call, right? | 18:06 |
| fungi | rcarrillocruz: yes | 18:06 |
| rcarrillocruz | cos i believe they're sharing things via Lync | 18:06 |
| pleia2 | rcarrillocruz: the "pages" are a bit confusing :) | 18:06 |
| fungi | and the pdf they included | 18:06 |
| rcarrillocruz | /facepalm | 18:06 |
| pleia2 | ah, I could use the pdf | 18:07 |
| *** Hal1 has quit IRC | 18:07 | |
| rcarrillocruz | oh, didn't notice that | 18:07 |
| rcarrillocruz | sweet | 18:07 |
| fungi | it was attached to the invite follow-up, but i can forward | 18:07 |
| fungi | pleia2: which address? | 18:07 |
| pleia2 | fungi: lyz@princessleia.com | 18:07 |
| rcarrillocruz | yeah, cos i'm not seeing, let me search | 18:07 |
| pleia2 | ty | 18:07 |
| pabelanger | fungi: smarcet okay, I see a stacktrace. safe to public on paste.o.o? | 18:08 |
| rcarrillocruz | got it | 18:08 |
| anteaya | pleia2: i also sent | 18:08 |
| fungi | pleia2: sent | 18:08 |
| smarcet | pabelanger: i think so | 18:08 |
| pleia2 | thanks | 18:08 |
| *** bpokorny has joined #openstack-infra | 18:08 | |
| fungi | pabelanger: in which logfile? | 18:08 |
| *** heyongli has quit IRC | 18:08 | |
| pabelanger | log-apache2handler-2016-03-22.txt | 18:09 |
| *** d0ugal has quit IRC | 18:09 | |
| pabelanger | http://paste.openstack.org/show/491492/ last stacktrace | 18:09 |
| *** heyongli has joined #openstack-infra | 18:09 | |
| fungi | pabelanger: that one's been happening for ages | 18:09 |
| smarcet | pabelanger : nope | 18:09 |
| smarcet | that is not the error | 18:09 |
| pabelanger | k | 18:09 |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Use shade for all OpenStack interactions https://review.openstack.org/226751 | 18:10 |
| smarcet | pabelanger seems something related to laravel fwk | 18:11 |
| fungi | pabelanger: smarcet: so the biggest concern i see in that file is the last entry is from 2 hours ago | 18:11 |
| fungi | is it possible there's a service which isn't running and should be? | 18:11 |
| smarcet | Set the debug option to true in app/config/app.php - t | 18:11 |
| fungi | smarcet: could this be fallout from the puppet change which merged to switch the "environment" from dev to production? | 18:11 |
| smarcet | fungi: it could | 18:12 |
| smarcet | but we need to turn out debug on to see actual error | 18:12 |
| *** samuelBartel has joined #openstack-infra | 18:12 | |
| pabelanger | clarkb: so, I am noticing the hostname on the centos-7 wheel mirror is a UUID, compared to wheel-mirror-ubuntu-trusty where is it not a UUID | 18:12 |
| clarkb | pabelanger: oh! ya thats the bug | 18:12 |
| *** SumitNaiksatam has joined #openstack-infra | 18:12 | |
| pabelanger | nice | 18:13 |
| fungi | pabelanger: were you adjusting app/config/app.php or should i? | 18:13 |
| pabelanger | fungi: no, I stopped at the moment | 18:13 |
| fungi | okay, i will | 18:13 |
| pabelanger | fungi: if you want me to take over, I can. | 18:13 |
| fungi | smarcet: i've switched debug from false to true in /srv/openstackid/w/app/config/app.php | 18:14 |
| pabelanger | clarkb: so, what should I be doing to fix it? Manually change or one of our ansible playbooks? | 18:14 |
| smarcet | Predis \ Connection \ ConnectionException | 18:14 |
| smarcet | Connection refused [tcp://127.0.0.1:6379] | 18:14 |
| clarkb | pabelanger: mordred pushed a thing, maybe just rebuild if thats easiest | 18:14 |
| smarcet | seems that redis is out of service? | 18:15 |
| pabelanger | clarkb: okay, I can do that | 18:15 |
| smarcet | wait | 18:15 |
| fungi | smarcet: yeah, i see nothing listening on 6379 on the loopback | 18:15 |
| smarcet | 6379 | 18:15 |
| smarcet | that is a wrong port | 18:15 |
| smarcet | :/ | 18:15 |
| smarcet | mmmmmm | 18:15 |
| smarcet | there were another change | 18:15 |
| smarcet | that was approved | 18:15 |
| fungi | should be 127.0.0.1:6378 ? | 18:15 |
| fungi | i see that listening | 18:15 |
| smarcet | fungi: someone refactored things | 18:16 |
| smarcet | :/ | 18:16 |
| pabelanger | clarkb: any preferred method to delete a server? | 18:16 |
| mordred | pabelanger: carefully | 18:17 |
| fungi | modules/openstack_project/manifests/openstackid_prod.pp has $redis_port = '6378' | 18:17 |
| pabelanger | mordred: :) | 18:17 |
| *** ybathia has joined #openstack-infra | 18:17 | |
| smarcet | fungi: yes i saw one change that saids tha refactored thing to be puppet complaint | 18:17 |
| *** ddieterly[away] has quit IRC | 18:17 | |
| jeblair | clarkb, pabelanger, mordred: ah, so the bug isn't really related to duplicate servers, it's any server? | 18:18 |
| clarkb | jeblair: looks like it | 18:18 |
| jeblair | will update storyboard | 18:18 |
| pabelanger | do you mind linking to the bug in question? | 18:18 |
| fungi | smarcet: in system-config or puppet-openstackid? i'm looking for that now | 18:18 |
| jeblair | pabelanger: https://storyboard.openstack.org/#!/story/2000520 | 18:18 |
| pabelanger | thanks | 18:19 |
| *** heyongli has quit IRC | 18:19 | |
| *** heyongli has joined #openstack-infra | 18:19 | |
| pabelanger | okay, will hold off until 295962 lands | 18:19 |
| smarcet | fungi: this one https://review.openstack.org/#/c/295320/3/manifests/init.pp | 18:19 |
| smarcet | but nm its doesnt break anything | 18:19 |
| smarcet | wonder where 6379 came | 18:20 |
| smarcet | fungi: there is a app/config/production folder on site root? | 18:21 |
| fungi | smarcet: i see it in puppet-redis | 18:21 |
| openstackgerrit | Merged openstack-infra/puppet-bugdaystats: Order of the class parameters is refactored https://review.openstack.org/294964 | 18:21 |
| mordred | pabelanger, jeblair is this new info as of when I wrote the patch? or just updating the story with words? | 18:21 |
| *** boris-42 has joined #openstack-infra | 18:21 | |
| fungi | smarcet: manifests/init.pp has $redis_port = '6379' | 18:21 |
| pabelanger | okay, wheel-mirror-centos-7 deleted | 18:21 |
| zaro | fungi, EmilienM : jobs are no longer unstable here: https://review.openstack.org/#/c/294803/ did the copy artifacts issue get fixed? | 18:21 |
| smarcet | fungi: could u tell me the content of bootstrap/environment.php file ? | 18:23 |
| pabelanger | mordred: jeblair: not sure how new, but I just launched a new server 45mins and UUID was present for hostname | 18:23 |
| jeblair | mordred: i believe it's new info, i don't think it changes the patch/fix | 18:23 |
| smarcet | fungi: also there exists a folder called app/config/production on site root? if so, which files its has ? | 18:23 |
| AJaeger | fungi, thanks for the followup on hexdump! | 18:23 |
| fungi | smarcet: http://paste.openstack.org/show/491495/ | 18:24 |
| *** ddieterly has joined #openstack-infra | 18:24 | |
| smarcet | fungi: ok that means that is looking for configurations on app/config/production folder | 18:24 |
| smarcet | there are files there ? | 18:24 |
| fungi | smarcet: the /srv/openstackid/w/app/config/production directory exists but is empty | 18:24 |
| smarcet | oh that is the problem then | 18:24 |
| AJaeger | anteaya: regarding your comment on https://review.openstack.org/295499 "layout is 7M" - that is normal. | 18:25 |
| fungi | so it was the dev->production environment switch | 18:25 |
| *** daemontool has quit IRC | 18:25 | |
| smarcet | fungi: yes it was | 18:25 |
| fungi | smarcet: should we revert that change for now? | 18:25 |
| anteaya | AJaeger: okay | 18:25 |
| jeblair | mordred: unless you think that 'server.id' should be the hostname iff there is only one server by that name. | 18:25 |
| smarcet | yes i think so | 18:25 |
| jeblair | mordred: if that's the case, then i think there is an underlying assumption that may be wrong. | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Use shade for all OpenStack interactions https://review.openstack.org/226751 | 18:25 |
| *** sdake_ is now known as sdake | 18:25 | |
| AJaeger | anteaya: you need to look at compare-xml file: http://logs.openstack.org/99/295499/3/check/gate-project-config-compare-xml/2effc7a/console.html#_2016-03-22_10_08_25_662 | 18:25 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Revert "Update enviroment id on openstackid production" https://review.openstack.org/295986 | 18:25 |
| fungi | smarcet: approving ^ that now | 18:25 |
| smarcet | ok | 18:25 |
| AJaeger | anteaya: "Number of changed jobs: 41" - so that shouldn't cause problems | 18:26 |
| *** ybathia has quit IRC | 18:26 | |
| anteaya | AJaeger: okay | 18:26 |
| mordred | jeblair: server.id should never be the hostname | 18:26 |
| mordred | jeblair: server.id will always be the UUID of the host | 18:26 |
| anteaya | sorry to bother | 18:26 |
| mordred | and has nothing to do with other hosts of the same name | 18:26 |
| jeblair | mordred: then i think we're all good | 18:26 |
| mordred | jeblair: cool | 18:26 |
| mordred | jeblair: I believe the problem is that we were hacking as if launch-node was going to be under the auspices of the dynamic inventory - and then also wrote out a per-server temporary static inventory - so by double solving the problem we made a nice new one | 18:27 |
| jeblair | mordred: yep. i think temp static inventory is a great solution. | 18:27 |
| jeblair | makes this very easy to reason about | 18:27 |
| mordred | jeblair: btw - this is another case where the micro-library I extracted from the zuul v3 branch could be used | 18:28 |
| mordred | "please to run this playbook with this isolated inventory" | 18:28 |
| smarcet | fungi: its weird bc https://github.com/openstack-infra/puppet-openstackid/blob/master/files/functions | 18:29 |
| mordred | (although I don't want to do more engineering to this script - more calling out the usecase as we think about whether or not that really does want to be a reusable lib or just part of zuul) | 18:29 |
| smarcet | specify copying files by $LARAVEL_ENV var | 18:29 |
| jeblair | mordred: ya | 18:29 |
| *** heyongli has quit IRC | 18:29 | |
| smarcet | that should be set to production | 18:29 |
| smarcet | :/ | 18:29 |
| *** heyongli has joined #openstack-infra | 18:29 | |
| AJaeger | mtreinish: could you confirm that "tox -e py27" needs databases setup for subunit2sql, please? See https://review.openstack.org/#/c/295044/ | 18:30 |
| *** ybathia has joined #openstack-infra | 18:30 | |
| *** bpokorny has quit IRC | 18:30 | |
| pleia2 | storyboard emailed me :D | 18:31 |
| dmsimard | What's the process behind uploading tarballs to tarballs.openstack.org ? Is there a jenkins job that picks up new tags or new commits and uploads them ? | 18:31 |
| AJaeger | woot, pleia2 ! | 18:31 |
| zaro | jeblair: did you ever get around to trying to fix the issue with Gerrit not being able to query some changes? | 18:31 |
| pleia2 | James E. Blair commented on "shade-launch-node... | 18:31 |
| pabelanger | rcarrillocruz: I remember the 2 NIcs too | 18:31 |
| AJaeger | dmsimard: that's a post job | 18:31 |
| rcarrillocruz | ++ | 18:31 |
| jeblair | zaro: nope | 18:31 |
| pabelanger | rcarrillocruz: get them wired up, and deal with network changes latter | 18:31 |
| pabelanger | later* | 18:31 |
| dmsimard | AJaeger: can you point me in the right direction ? | 18:31 |
| zaro | dmsimard: yes, that's how it works | 18:31 |
| *** dahoo has joined #openstack-infra | 18:31 | |
| anteaya | pleia2: yay! | 18:31 |
| AJaeger | dmsimard: {name}-tarball in jenkins/jobs/python-jobs.yaml | 18:31 |
| dmsimard | AJaeger: cool, thanks | 18:31 |
| *** shardy has quit IRC | 18:32 | |
| *** dahoo is now known as jerryz | 18:32 | |
| AJaeger | dmsimard: do you want to use it for your repo? Which one? | 18:32 |
| dmsimard | AJaeger: Would packstack be allowed in there or is it for big tent only ? | 18:32 |
| *** apevec has joined #openstack-infra | 18:32 | |
| dmsimard | but I'm just looking at the general process right now | 18:32 |
| AJaeger | dmsimard: you can use that. What do you want to upload? | 18:33 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 18:33 | |
| AJaeger | dmsimard: manila uploads also images... | 18:33 |
| zaro | jeblair: you got my suggestions on things to try? | 18:33 |
| *** jed56 has quit IRC | 18:33 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/release-tools: Added tool to reset backport-potential tags once backports are merged https://review.openstack.org/295993 | 18:34 |
| jeblair | zaro: i don't recall | 18:34 |
| dmsimard | AJaeger: the source, for packaging purposes | 18:34 |
| mgagne | anyone having issue running ansible in a dib element on Ubuntu Trusty? I get this error: http://paste.openstack.org/show/491497/ | 18:34 |
| dmsimard | AJaeger: so, just like any python project tarball, I guess | 18:35 |
| mgagne | once I move old /dev/shm (which is a symlink to /run/shm) and mount a new one (tmpfs), it works fine | 18:35 |
| apevec | dmsimard, what was the pointer, are we missing something in tox.ini ? | 18:35 |
| dmsimard | apevec: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/python-jobs.yaml#L392-L413 | 18:35 |
| dmsimard | apevec: packstack doesn't have a post -tarball job | 18:36 |
| *** fawadkhaliq has joined #openstack-infra | 18:36 | |
| openstackgerrit | Major Hayden proposed openstack-infra/shade: Use profile instead of cloud https://review.openstack.org/295995 | 18:36 |
| AJaeger | dmsimard: check also the usage in zuul/layout.yaml, look for openstack-server-release-jobs | 18:36 |
| dmsimard | AJaeger: yeah that's what I'm looking at right now | 18:36 |
| *** jtomasek has quit IRC | 18:37 | |
| openstackgerrit | Merged openstack-infra/system-config: Don't set the host's name as UUID https://review.openstack.org/295962 | 18:37 |
| *** tongli has joined #openstack-infra | 18:37 | |
| anteaya | igorbelikov: I see you have an item on the infra meeting agenda under open discussion | 18:37 |
| doug-fish | Looks like the zanata instance is unavailable again https://translate.openstack.org/ | 18:37 |
| doug-fish | are there openstackid issues? | 18:37 |
| zaro | jeblair: i would suggest 2 things. fix change (https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#fix-change) and flush caches. | 18:38 |
| anteaya | igorbelikov: we tend to put meeting items on the agenda, rather than in open discussion | 18:38 |
| dmsimard | AJaeger: oh so that's where all these release emails come from :) | 18:38 |
| smarcet | doug-fish: yes we are facing a outage | 18:38 |
| smarcet | working on that :) | 18:38 |
| anteaya | igorbelikov: I know other team put agenda items in open discussion so it can be confusing | 18:38 |
| jeblair | zaro: do you have a list of those changes? | 18:38 |
| doug-fish | smarcet: understood - thanks! | 18:38 |
| jeblair | zaro: i don't have a lot of time to work on this on my own, but if you can send me the commands to run, i can run them. or any other infra-root. | 18:39 |
| EmilienM | zaro: all is clear to me, thanks to fungi | 18:39 |
| *** heyongli has quit IRC | 18:39 | |
| *** heyongli has joined #openstack-infra | 18:39 | |
| *** vgridnev has quit IRC | 18:40 | |
| *** angdraug has joined #openstack-infra | 18:40 | |
| rcarrillocruz | k | 18:41 |
| rcarrillocruz | now run | 18:41 |
| rcarrillocruz | not sure i'll be on time for infra meeting | 18:41 |
| rcarrillocruz | gotta drive back home | 18:42 |
| rcarrillocruz | later | 18:42 |
| anteaya | rcarrillocruz: drive safe | 18:42 |
| fungi | smarcet: sorry, was trying to juggle attention with a conference call, but that just ended so i'm seeing what else we can do to speed up the resolution now | 18:42 |
| clarkb | I am not sure I caught the comment about the ecopod ownership, did I mishear or were they saying it might be taken over by the other hp entity in the future? | 18:42 |
| *** vgridnev has joined #openstack-infra | 18:42 | |
| doug-fish | smarcet: is there any estimated time for how long it might take to fix? It's near the end of our translators day; maybe time to just send them home | 18:42 |
| smarcet | fungi: sure and thx | 18:42 |
| smarcet | doug-fish: waiting to deploy a fix | 18:43 |
| *** ihrachys has quit IRC | 18:43 | |
| fungi | looks like 295986 is a few minutes from merging still but i'll try to apply locally while it's finishing up | 18:43 |
| pabelanger | okay, re-creating the centos-7 wheel mirror now | 18:43 |
| *** HeOS has quit IRC | 18:43 | |
| zaro | jeblair: ok, will git it up. | 18:44 |
| jeblair | zaro: thanks | 18:44 |
| clarkb | pabelanger: ok | 18:44 |
| doug-fish | smarcet: any idea how long the wait is? - not trying to harass you, just trying to plan | 18:44 |
| doug-fish | ... I have no guess what you are waiting for | 18:45 |
| anteaya | doug-fish: I think fungi said he is hand applying the change | 18:45 |
| doug-fish | I see | 18:45 |
| *** _ari_ has quit IRC | 18:45 | |
| doug-fish | so that implies a fix may arrive soon, like < 1hr? | 18:45 |
| *** _ari_ has joined #openstack-infra | 18:45 | |
| fungi | yeah, we're waiting for 295986 to merge, which has one job left estimated at 4 minutes, but it can take puppet a good 15 minutes to apply it once merged so i'm tracing through the manifests right now to see what files need editing to have the same effect | 18:46 |
| doug-fish | got it. Thanks! | 18:46 |
| fungi | looks like environment.php and openstackid.conf | 18:48 |
| clarkb | pabelanger: in grafana is there any way to make it not change scales? It is really annoying that it goes from minutes to hours on the same axis | 18:48 |
| openstackgerrit | Merged openstack-infra/system-config: Revert "Update enviroment id on openstackid production" https://review.openstack.org/295986 | 18:48 |
| pabelanger | clarkb: I think so, which panel are you looking at? | 18:49 |
| clarkb | pabelanger: the devstack runtime graph on the nodepool* dashboards | 18:49 |
| fungi | smarcet: doug-fish: it looks like just editing environment.php temporarily was sufficient. can you test and confirm? | 18:49 |
| *** heyongli has quit IRC | 18:49 | |
| doug-fish | will do! | 18:50 |
| smarcet | yes | 18:50 |
| doug-fish | yep, that did it | 18:50 |
| smarcet | its works | 18:50 |
| doug-fish | thanks! | 18:50 |
| smarcet | thx | 18:50 |
| *** heyongli has joined #openstack-infra | 18:50 | |
| pabelanger | clarkb: so, I think you can force it to be a specific time period and unit of time | 18:50 |
| smarcet | and sorry for disruptiom | 18:50 |
| pabelanger | need to double check the syntax | 18:50 |
| doug-fish | understood - thanks for the quick fix! | 18:50 |
| *** arif-ali has quit IRC | 18:50 | |
| smarcet | fungi: tnx ! | 18:50 |
| fungi | smarcet: no apology needed, glad we found the issue. just trying to get it resolved quickly since i need to chair a meeting in 10 minutes | 18:51 |
| dmsimard | AJaeger: I don't think we want to use openstack-server-release-jobs verbatim, mainly because we'll start without the announce and merge tags macros. How are the pre-release and release gates triggered ?% | 18:51 |
| *** dkliban has joined #openstack-infra | 18:51 | |
| fungi | crinkle: rcarrillocruz: sorry i probably didn't pay as close attention in that call as i wanted. was someone going to write up a brief summary? | 18:51 |
| anteaya | fungi: crinkle has posted to the ml | 18:52 |
| crinkle | fungi: I sent some notes to the list | 18:52 |
| pabelanger | okay, hostname looks better for wheel-mirror-centos-7 | 18:52 |
| anteaya | crinkle: thank you for that | 18:52 |
| fungi | crinkle: wow, so fast! | 18:52 |
| pleia2 | thanks crinkle | 18:52 |
| AJaeger | dmsimard: release gates: When you tag your repo. I think we do not need pre-release anymore | 18:53 |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: don't execute funct jobs when *.json patch https://review.openstack.org/295824 | 18:53 |
| dmsimard | AJaeger: ok so I just push a tag to gerrit and that will trigger the release gate ? | 18:53 |
| clarkb | ya I think the biggest reason to 10Gbe is that our images are in the 6GB range andonce a day every hypervisor needs to grab 5 of them | 18:53 |
| *** IlyaG has quit IRC | 18:54 | |
| clarkb | 30GB * 100 hypervisors = use 10Gbe | 18:54 |
| AJaeger | dmsimard: yes, for details read also the infra manual | 18:54 |
| fungi | clarkb: speaking of images and bandwidth... 200mbps uplink :/ | 18:54 |
| jeblair | AJaeger, dmsimard: terminology nit: the "pre-release" and "release" pipelines aren't gates. | 18:54 |
| AJaeger | jeblair: indeed ;) | 18:54 |
| dmsimard | jeblair: good catch, thanks :) | 18:54 |
| AJaeger | jeblair: do we use pre-release still? | 18:54 |
| *** ybathia has quit IRC | 18:55 | |
| AJaeger | anteaya: could you review later https://review.openstack.org/#/c/295046/ and https://review.openstack.org/#/c/295435/ , please? Those are two python db changes | 18:56 |
| anteaya | I will | 18:56 |
| jeblair | AJaeger: i don't know how that may have changed with respect to git-based release process, sorry | 18:56 |
| *** tosky has quit IRC | 18:57 | |
| AJaeger | thanks, anteaya | 18:57 |
| fungi | i think most projects have ceased pushing pre-release tags | 18:57 |
| jhesketh | Morning | 18:57 |
| AJaeger | jeblair: mmh, it is triggered by a regex - *(a|b|rc)[0-9]+$ - so, should have been used by rc1... | 18:58 |
| AJaeger | so, answering my question: It looks to be still needed... | 18:58 |
| *** mylu has joined #openstack-infra | 18:58 | |
| fungi | oh, right we do use rc tags on stuff that doesn't generally get pushed to pypi at the moment | 18:58 |
| *** Sukhdev has joined #openstack-infra | 18:59 | |
| mgagne | a coworker found that DIB mount --bind /dev into the image. Ubuntu Trusty however has a symlink from /dev/shm to /run/shm and we suspect the image fails to find the symlink target: https://github.com/openstack/diskimage-builder/blob/master/lib/common-functions#L279 | 18:59 |
| AJaeger | Ah - nova has 12.0.0.0b3 and 12.0.0.0rc1 etc | 18:59 |
| mgagne | how do you suggest we fix this? mount --bind /run too? | 18:59 |
| AJaeger | fungi, that explains it. | 18:59 |
| anteaya | AJaeger: welcome | 18:59 |
| anteaya | jhesketh: morning | 18:59 |
| fungi | it's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour | 18:59 |
| *** heyongli has quit IRC | 19:00 | |
| igorbelikov | anteaya: thanks, I moved it out of open discussion | 19:00 |
| anteaya | igorbelikov: awesome thank you | 19:00 |
| *** heyongli has joined #openstack-infra | 19:00 | |
| AJaeger | anteaya: https://review.openstack.org/#/c/295435/ has the same commit message ;( But I'm not changing this anymore unless I need to rebase if that's ok for you. | 19:01 |
| anteaya | AJaeger: yeah that's fine, just a nit | 19:01 |
| *** apevec has left #openstack-infra | 19:02 | |
| *** arxcruz has quit IRC | 19:02 | |
| *** EricGonczer_ has quit IRC | 19:02 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Fix the path to get_host_info.sh https://review.openstack.org/295913 | 19:02 |
| *** EricGonczer_ has joined #openstack-infra | 19:03 | |
| *** kzaitsev_mb has quit IRC | 19:03 | |
| *** mtanino has quit IRC | 19:04 | |
| dmsimard | mgagne: maybe someone from #tripleo would know | 19:04 |
| *** dizquierdo has quit IRC | 19:04 | |
| *** jsavak has quit IRC | 19:06 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci: WIP: Use IPv6 on the gate jobs https://review.openstack.org/289445 | 19:06 |
| smarcet | fungi: | 19:06 |
| smarcet | https://openstackid.org | 19:06 |
| smarcet | is down again | 19:06 |
| *** jsavak has joined #openstack-infra | 19:07 | |
| dmsimard | That's a pretty error page. | 19:07 |
| *** mylu has quit IRC | 19:07 | |
| smarcet | that is bc its on debug mode ;) | 19:07 |
| *** achanda has quit IRC | 19:07 | |
| *** hashar has joined #openstack-infra | 19:07 | |
| *** mylu has joined #openstack-infra | 19:07 | |
| *** jamesmcarthur has joined #openstack-infra | 19:08 | |
| smarcet | jeblair: could u help me with openstackid.org? | 19:08 |
| smarcet | nm is up again | 19:09 |
| jamesmcarthur | jeblair: fungi: clarkb: we’re having some pretty serious outages with openstackid. I know smarcet has been working with fungi all day. Could use some help understanding why the server is up and down. | 19:09 |
| jamesmcarthur | Any insight? | 19:09 |
| pleia2 | the infra team meeting is happening right now | 19:10 |
| *** heyongli has quit IRC | 19:10 | |
| fungi | jamesmcarthur: smarcet proposed a change to update the laravel environment from dev to production, that's been reverted and also manually applied | 19:10 |
| *** heyongli has joined #openstack-infra | 19:10 | |
| jamesmcarthur | ok - thanks sir | 19:11 |
| fungi | smarcet: had a bunch of adjustments he wanted to make to the deployment tooling before we tried to build a larger replacement to handle the laod projections he and jpmaxman have | 19:11 |
| fungi | load projections | 19:11 |
| *** ihrachys has joined #openstack-infra | 19:12 | |
| *** electrofelix has quit IRC | 19:12 | |
| fungi | smarcet: i suspect there was a very brief window where my manual application of the fix got undone by puppet because it had started running with the previous version of the repo from before that change merged, but then corrected itself on the next run when it had the updated version wit the revert | 19:12 |
| jpmaxman | thanks for your help fungi :) | 19:12 |
| smarcet | fungi: tnx for the info | 19:12 |
| fungi | and yeah, the outage yesterday was due to lack of idempotency on the deployment tooling. it assumed it should always initialize the database | 19:13 |
| fungi | one of smarcet's changes merged today should solve that | 19:13 |
| *** ihrachys has quit IRC | 19:13 | |
| AJaeger | doug-fish, smarcet, I can log into translate.openstack.org! | 19:14 |
| doug-fish | AJaeger: smarcet and fungi fixed it quickly! | 19:15 |
| *** ihrachys has joined #openstack-infra | 19:15 | |
| *** achanda has joined #openstack-infra | 19:16 | |
| *** ociuhandu has quit IRC | 19:16 | |
| *** ihrachys has quit IRC | 19:17 | |
| *** arif-ali has joined #openstack-infra | 19:17 | |
| *** ddieterly is now known as ddieterly[away] | 19:17 | |
| jamesmcarthur | thanks fungi: smarcet filled me in as well :) | 19:20 |
| *** heyongli has quit IRC | 19:20 | |
| *** heyongli has joined #openstack-infra | 19:21 | |
| *** ddieterly[away] is now known as ddieterly | 19:22 | |
| SamYaple | hey people. we got some spam here https://wiki.openstack.org/wiki/How_To_Contribute | 19:24 |
| SamYaple | not sure how to proceed wit hthat | 19:24 |
| pleia2 | SamYaple: pabelanger is looking into cleanup (there's a thread on the openstack-infra mailing list) | 19:25 |
| SamYaple | pleia2: awesome. i cant keep up with all the threads! | 19:25 |
| SamYaple | thanks | 19:25 |
| pabelanger | clarkb: so... hostname is set as wheel-mirror-centos-7-amd64 in /etc/hostname | 19:26 |
| pabelanger | clarkb: but should be wheel-mirror-centos-7-amd64.slave.openstack.org | 19:27 |
| pabelanger | I think our set_hostname playbook is wrong | 19:27 |
| *** mylu has quit IRC | 19:27 | |
| pleia2 | SamYaple: I just restored to the last good, known version for that particular page | 19:27 |
| pabelanger | err.. role | 19:27 |
| clarkb | pabelanger: I think hostname is supposed to be short but hostname -f returns the full fqdn | 19:27 |
| *** ihrachys has joined #openstack-infra | 19:27 | |
| pabelanger | $ hostname -f | 19:28 |
| pabelanger | hostname: Name or service not known | 19:28 |
| pabelanger | on centos-7 mirror | 19:28 |
| SamYaple | clarkb: ++ /etc/hostname is short | 19:28 |
| SamYaple | hostname -f gives fullqualified if nss stuff is setup correctly | 19:28 |
| *** tonytan4ever has quit IRC | 19:29 | |
| pabelanger | so, we have some conflicts then because wheel-mirror-ubuntu-trusty-amd64.slave.openstack.org is showing in /etc/hostname | 19:29 |
| pabelanger | on ubuntu-trusty | 19:29 |
| SamYaple | pabelanger: yea thats not correct, but its is a common mistake | 19:29 |
| *** tonytan4ever has joined #openstack-infra | 19:29 | |
| *** fawadkhaliq has quit IRC | 19:30 | |
| *** mylu has joined #openstack-infra | 19:30 | |
| *** fawadkhaliq has joined #openstack-infra | 19:30 | |
| pabelanger | and /etc/hosts is missing alias for wheel-mirror-centos-7-amd64 | 19:30 |
| *** heyongli has quit IRC | 19:30 | |
| openstackgerrit | Alan Pevec proposed openstack/requirements: WIP test cryptograhy 1.3.1 update https://review.openstack.org/296019 | 19:30 |
| *** heyongli has joined #openstack-infra | 19:31 | |
| SamYaple | /etc/hostname should short name, /etc/hosts should have short and fqdn (or upstream dns should resolve that stuff) | 19:32 |
| clarkb | well you want a host to know its fqdn too | 19:32 |
| clarkb | so you edit /etc/hosts to include that | 19:32 |
| SamYaple | its not a requirement technically, but yea | 19:32 |
| SamYaple | a host can know its own by asking in a proper dns setup | 19:33 |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/tripleo-ci: Allow the container job to run again https://review.openstack.org/288915 | 19:33 |
| clarkb | ya reverse dns is so iffy though. Like we can't get it working in any of the non rax clouds I don't think | 19:33 |
| pabelanger | so, there must be a bug in launch-node when using set_hostnames playbook | 19:33 |
| SamYaple | i have forgotted alot of the nss stuff but im pretty sure as long as it resolves from teh stack its fine | 19:33 |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Test centos-7 with earlier glean start" https://review.openstack.org/294363 | 19:33 |
| SamYaple | yea setting it in /etc/hosts is the way to go | 19:33 |
| *** mylu has quit IRC | 19:34 | |
| *** ociuhandu has joined #openstack-infra | 19:35 | |
| openstackgerrit | Rossella Sblendido proposed openstack-infra/project-config: Add job to update the Neutron project dashboard https://review.openstack.org/284284 | 19:36 |
| mordred | SamYaple, clarkb: so we should be setting /etc/hosts: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/roles/set_hostname/templates/hosts.j2 | 19:36 |
| mordred | here's the full playbook: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/roles/set_hostname/tasks/main.yml#n6 | 19:36 |
| SamYaple | yea that looks right | 19:36 |
| pabelanger | mordred: we only do that with launch-node I think. I think we might need to add it to run_all.sh, to avoid drift | 19:37 |
| pabelanger | becuase these 2 servers I am looking at are not setup like set_hostname | 19:37 |
| *** harlowja_at_home has quit IRC | 19:37 | |
| openstackgerrit | Rossella Sblendido proposed openstack-infra/project-config: Add job to update the Neutron project dashboard https://review.openstack.org/284284 | 19:38 |
| openstackgerrit | Merged openstack-infra/project-config: Fix MAGNUM_IMAGE_NAME setting https://review.openstack.org/295969 | 19:38 |
| mordred | pabelanger: we will need to change it some if we want to use it in run_all | 19:38 |
| mordred | pabelanger: otherwise we will have the uuid problem for duplicate names | 19:38 |
| mordred | pabelanger: but totally doable - and I agree that we should | 19:38 |
| pabelanger | http://paste.openstack.org/show/491503/ newest server, launched 30mins ago | 19:39 |
| pabelanger | not what I would expect based on launch-node set_hostname playbook | 19:39 |
| mordred | me neither | 19:40 |
| *** heyongli has quit IRC | 19:41 | |
| *** heyongli has joined #openstack-infra | 19:41 | |
| *** kzaitsev_mb has joined #openstack-infra | 19:42 | |
| *** mylu has joined #openstack-infra | 19:42 | |
| *** egonzalez has joined #openstack-infra | 19:44 | |
| *** mylu has quit IRC | 19:44 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 19:44 | |
| *** dkliban has left #openstack-infra | 19:45 | |
| *** mylu has joined #openstack-infra | 19:47 | |
| *** dtardivel has quit IRC | 19:48 | |
| sdake | i have a quick question | 19:51 |
| sdake | are kolla requests on irc more or less annoying then any other project | 19:51 |
| *** heyongli has quit IRC | 19:51 | |
| sdake | i mean, do wesuck up too much time on openstack-infra | 19:51 |
| * Clint chuckles. | 19:51 | |
| jroll | hahaha | 19:51 |
| *** heyongli has joined #openstack-infra | 19:51 | |
| sdake | openstack-inra has like a million pmessasgs a day so i don't watch it closely - so hard to tell the load we are putting on infra | 19:52 |
| sdake | from a support perspetive, not code perspective | 19:52 |
| *** mylu has quit IRC | 19:52 | |
| openstackgerrit | Rob Crittenden proposed openstack-infra/project-config: Add experimental job to test TLS proxy in devstack https://review.openstack.org/293090 | 19:53 |
| *** imcsk8 has joined #openstack-infra | 19:53 | |
| *** bpokorny has joined #openstack-infra | 19:54 | |
| pabelanger | clarkb: okay, I've manually fixed hostnames for now, but now I need to update hiera to include the wheel_keytab for centos-7 mirror | 19:54 |
| pabelanger | I've not done hiera yet | 19:54 |
| hashar | clarkb: zaro: for Wikimedia, most people ends up cloning from the Github mirror then change the remote to our Gerrit | 19:55 |
| clarkb | pabelanger: its on puppet master on /etc/puppet/hieradata/production iirc. Check git status make sure no one else has edits (to avoid conflicts), then edit the host specific file and git commit it | 19:55 |
| hashar | clarkb: zaro: mediawiki/core is quite large (maybe 300-400Mbytes), and github is significantly faster than Gerrit in serving it | 19:55 |
| clarkb | pabelanger: usually we sign off with our name/nick in the commit messages since otherwise its all just root | 19:55 |
| *** piet has joined #openstack-infra | 19:56 | |
| *** piet has quit IRC | 19:56 | |
| *** david-lyle_ has joined #openstack-infra | 19:56 | |
| *** kgiusti has quit IRC | 19:57 | |
| clarkb | hashar: yes we don't tell people to use gerrit, we point them at git.o.o instead | 19:57 |
| clarkb | hashar: but I think the plan would be to run the same gc/repack setup in gerrit and the git mirrors | 19:57 |
| *** tonytan_brb has joined #openstack-infra | 19:57 | |
| *** david-lyle has quit IRC | 19:57 | |
| *** ekarlso- has quit IRC | 19:58 | |
| *** jeblair has quit IRC | 19:58 | |
| *** salv-orlando has quit IRC | 19:58 | |
| *** jeblair has joined #openstack-infra | 19:58 | |
| *** jlvillal has quit IRC | 19:58 | |
| *** pots has quit IRC | 19:58 | |
| *** Mmike has quit IRC | 19:58 | |
| anteaya | cdent: thanks for participating, sorry we didnt' get to have a full discussion | 20:00 |
| cdent | pabelanger, anteaya: if you guys want me to do anything let me know | 20:00 |
| *** tonytan4ever has quit IRC | 20:01 | |
| anteaya | cdent: sure I think we haven't had a lot of discussions and ero not much policy around this | 20:01 |
| hashar | clarkb: zero: maybe reach out to Gerrit / Git folks? They might have some handy doc about maintaining huge repos | 20:01 |
| anteaya | cdent: so first of all just trying to get a sense of how large the spectrum is and then what we want | 20:01 |
| *** heyongli has quit IRC | 20:01 | |
| *** mylu has joined #openstack-infra | 20:01 | |
| anteaya | cdent: thanks for being willing to participate and help :) | 20:01 |
| *** heyongli has joined #openstack-infra | 20:01 | |
| *** david-lyle_ is now known as david-lyle | 20:02 | |
| cdent | anteaya: Yeah, I just want to avoid the issue that came up: people becoming dependent and my server/services being uncared for | 20:02 |
| anteaya | yup, sounds reasonable | 20:02 |
| anteaya | I think we might need some more discussion before we coalese on action | 20:02 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 20:02 |
| anteaya | but yeah, thank you :) | 20:02 |
| *** dprince has quit IRC | 20:02 | |
| *** baoli has quit IRC | 20:02 | |
| cdent | anteaya: no worries, I'll try to keep track. If it seems like I missed somethign please keep me informed. | 20:03 |
| *** Mmike has joined #openstack-infra | 20:03 | |
| *** jlvillal has joined #openstack-infra | 20:03 | |
| hashar | cdent: anteaya: hello! Regarding IRC bot, maybe you can get them hosted on a dedicated tenant were bots operators can share maintenance of all bots? Ie without requiring shell or root on the rest of the infra | 20:03 |
| *** kzaitsev_mb has quit IRC | 20:03 | |
| anteaya | cdent: yup, will do, thanks | 20:03 |
| *** nhandler_ has joined #openstack-infra | 20:03 | |
| *** nhandler has quit IRC | 20:04 | |
| hashar | at Wikimedia we have a shared tenant 'tool' with volunteers having root for maintenance and groups of users per bots. So it is fairly easy to add a person as a maintainer of a given bot / all bot or help maintain the shared tenant | 20:04 |
| *** ybathia has joined #openstack-infra | 20:04 | |
| anteaya | hashar: I'm not sure what the ramifications on that would be | 20:04 |
| hashar | which scales the grunt work out of the actual infrastructure roots (they deal with maintaining the tenant though) | 20:04 |
| *** Sukhdev has quit IRC | 20:05 | |
| hashar | so if you get Sarah willing to host a bot, she could join an 'ircbot' tenant, get her bot installed there and the maintenance can be shared among other users | 20:05 |
| hashar | without infra team having to take care of them (in theory) | 20:05 |
| hashar | so Emily maintaining another bot could help maintaining Sarah's bot even if Emily does not have shell access on the rest of the infra. She would just need access on the tenant running the bots | 20:06 |
| hashar | but maybe that does not apply for your use case :-( | 20:06 |
| *** nhandler_ is now known as nhandler | 20:06 | |
| *** rockyg has joined #openstack-infra | 20:07 | |
| anteaya | following the tc meeting, will read in a bit | 20:08 |
| pabelanger | clarkb: so, I think I might have found a bug. For some reason, the wheel-mirror.yaml file is not getting copied over to the centos-7 wheel-mirror, even though in the proper group. | 20:09 |
| *** sc68cal has quit IRC | 20:09 | |
| pabelanger | clarkb: if I grep the logs for ubuntu-trusty, I don't see it being copied over either | 20:09 |
| *** ekarlso- has joined #openstack-infra | 20:11 | |
| AJaeger | REgarding database access for unit tests, could I get some reviews for https://review.openstack.org/#/c/295035/ https://review.openstack.org/#/c/295435/ https://review.openstack.org/#/c/295044/ https://review.openstack.org/#/c/294949/ ,please ? I would then approve these tomorrow morning UTC... | 20:11 |
| *** heyongli has quit IRC | 20:11 | |
| *** heyongli has joined #openstack-infra | 20:12 | |
| pabelanger | clarkb: fungi is there an easy way to look at the current ansible inventory on the puppetmaster? | 20:12 |
| fungi | pabelanger: view /var/cache/ansible-inventory/ansible-inventory.cache | 20:13 |
| pabelanger | fungi: thank you | 20:13 |
| fungi | pabelanger: there's also `sudo shade-inventory --list` | 20:13 |
| fungi | to see what shade sees specifically | 20:14 |
| fungi | though the ansible inventory cache is likely what you want | 20:14 |
| *** kgiusti has joined #openstack-infra | 20:15 | |
| *** egonzalez has quit IRC | 20:16 | |
| pabelanger | fungi: so, if I am reading https://github.com/openstack-infra/ansible-puppet/blob/master/tasks/main.yml#L17 correctly, does that map to the $group var in site.pp? | 20:16 |
| *** sc68cal has joined #openstack-infra | 20:17 | |
| fungi | pabelanger: it tries to. there are two places to configure a host group | 20:18 |
| fungi | pabelanger: in system-config we also have modules/openstack_project/files/puppetmaster/groups.txt | 20:19 |
| pabelanger | fungi: right, so I would expect group_names to exist in some form in our inventory cache but failing to see it | 20:20 |
| mordred | hashar: we don't need a tenant for that - we can give out root to specific people on specific vms and currently do | 20:20 |
| mordred | pabelanger: look at the lists of hosts | 20:21 |
| mordred | pabelanger: like this: "review.openstack.org": [ | 20:22 |
| *** heyongli has quit IRC | 20:22 | |
| mordred | "afb0cdb2-4800-4268-af83-0b2d5f68a635", | 20:22 |
| mordred | "87741fa9-92cb-4d1e-8195-6841e2468fb2" | 20:22 |
| mordred | ] | 20:22 |
| *** heyongli has joined #openstack-infra | 20:22 | |
| mordred | pabelanger: that defines a group named review.openstack.org with two members, afb0cdb2-4800-4268-af83-0b2d5f68a635 and 87741fa9-92cb-4d1e-8195-6841e2468fb2 | 20:22 |
| *** IlyaG has joined #openstack-infra | 20:22 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove db set up from python unittests https://review.openstack.org/296039 | 20:23 |
| zaro | hashar: so all fetches come from github for your users? | 20:23 |
| bswartz | anyone know the timeframe when xenial will be available in nodepool? | 20:24 |
| clarkb | after its released and dib builds and glean work on it | 20:24 |
| anteaya | bswartz: we have an etherpad about operating system upgrades: https://etherpad.openstack.org/p/infra-operating-system-upgrades | 20:25 |
| fungi | bswartz: ubuntu isn't even releasing xenial until next month | 20:25 |
| *** mylu has quit IRC | 20:25 | |
| fungi | but hopefully shortly thereafter | 20:25 |
| bswartz | okay so you don't need to wait until xenial becomes LTS in July? | 20:26 |
| fungi | in the meantime, we can work on trying to get something xenial-like available (mordred did the legwork on getting dib builds of it booting such that we could use that in nodepool) | 20:26 |
| pabelanger | mordred: so, I don't think I am seeing that for wheel-mirror group. | 20:26 |
| pabelanger | mordred: which is failing to then copy the wheel-mirror.yaml file in hieradata | 20:27 |
| mordred | pabelanger: where have you defined that? | 20:27 |
| clarkb | bswartz: xenial is lts wgen it releases | 20:27 |
| pabelanger | mordred: site.pp? and the $group var? | 20:27 |
| mordred | ah - wheel-mirror *wheel-mirror-*.openstack.org | 20:27 |
| bswartz | trusty didn't become LTS until July 2014 (14.04.1) | 20:27 |
| fungi | bswartz: we won't be moving any of our servers to run on xenial until it's stabilized, but we hope to have it available in nodepool for running jobs shortly after the summit (hopefully newton will be officially released as tested on xenial for its primary ubuntu platform) | 20:27 |
| pabelanger | mordred: I have to admit, this is all new to me | 20:27 |
| mordred | pabelanger: yah - it's a new thing | 20:27 |
| mordred | so - groups are defined in groups.txt | 20:28 |
| bswartz | I'm presuming there will be a 16.04.1 which will be the first xenial LTS release | 20:28 |
| mordred | pabelanger: ./modules/openstack_project/files/puppetmaster/groups.txt in system-config | 20:28 |
| bswartz | okay that works for me | 20:28 |
| pabelanger | mordred: right, I see that | 20:28 |
| bswartz | as long as it's supported well before feature freeze it means we can expect to depend on it for gate jobs | 20:28 |
| clarkb | bswartz: thats not how lts works aiui | 20:29 |
| *** vgridnev has quit IRC | 20:29 | |
| pabelanger | mordred: which allows ansible-puppet to then push specific group heiradata files to hosts, if I understand | 20:29 |
| mordred | pabelanger: you might need to run modules/openstack_project/files/puppetmaster/expand-groups.sh | 20:29 |
| mordred | oh - one sec | 20:29 |
| clarkb | the point releases are an optimization | 20:29 |
| mordred | I have given you incomplete information | 20:29 |
| pabelanger | k | 20:29 |
| *** _ari_ has quit IRC | 20:29 | |
| *** mylu has joined #openstack-infra | 20:30 | |
| mordred | pabelanger: SO there is a bug somewhere | 20:30 |
| *** fawadkhaliq has quit IRC | 20:30 | |
| mordred | pabelanger: /etc/ansible/hosts/generated-groups should hold the expansions of the groups in groups.txt | 20:30 |
| *** fawadkhaliq has joined #openstack-infra | 20:30 | |
| bswartz | clarkb: my experience was that I tried to run do-release-upgrade from precise in April 2014 and nothing happened -- the ubuntu folks told me that 14.04.0 was considered a non-lts release and do-release-upgrade from an LTS release will only upgrade to another LTS release, so I had to wait for 14.04.1 | 20:30 |
| mordred | pabelanger: you can see a properly formed verison in /etc/ansible/hosts/generated-groups~ | 20:30 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: ssh less https://review.openstack.org/295973 | 20:31 |
| *** sdague has quit IRC | 20:31 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:31 | |
| pabelanger | mordred: Ah, okay. I see the diff now | 20:32 |
| *** heyongli has quit IRC | 20:32 | |
| *** heyongli has joined #openstack-infra | 20:32 | |
| mordred | pabelanger: so I'm guessing expand-groups had a sad | 20:33 |
| mordred | pabelanger: might be worth running it by hand and seeing what it does | 20:33 |
| pabelanger | mordred: as root I assume? | 20:33 |
| *** achanda has quit IRC | 20:34 | |
| *** catherineD has joined #openstack-infra | 20:34 | |
| clarkb | bswartz: for upgrades they dont jump until .1 but a .0 install cam be cared for with package updates for 5 years without doing anything special | 20:34 |
| *** zeih has joined #openstack-infra | 20:34 | |
| mordred | pabelanger: yah | 20:34 |
| pabelanger | mordred: okay, looks to have workedf | 20:35 |
| mordred | yes. I agree | 20:35 |
| mordred | which is sad - I would like to have known why it didn't work before | 20:35 |
| pabelanger | So, I did have some warnings from launch-shade, let me see if I can find them again | 20:35 |
| mordred | pabelanger: you know what ... | 20:36 |
| pabelanger | mordred: http://paste.openstack.org/show/491481/ | 20:36 |
| pabelanger | that was some of the error / warnings I got | 20:36 |
| openstackgerrit | Merged openstack-infra/shade: Use direct requests for flavor extra_specs set/unset https://review.openstack.org/295388 | 20:37 |
| mordred | pabelanger: oh - my - that's very weird | 20:37 |
| pabelanger | mordred: so, I ran as my local user, not sudo su -. Could that be the issue? | 20:38 |
| clarkb | bswartz: likely the biggest hurdle to get over will be things that systemd breaks | 20:38 |
| pabelanger | mordred: also, puppet is now working as I expect it on wheel-mirror-centos-7-amd64 | 20:38 |
| clarkb | possible glean issues and services devstack relies on not working quite right | 20:38 |
| mordred | clarkb: specifically, dib currently makes some 'detect systemd vs. upstart' assumptions that might be wrong in an ubuntu-systemd world | 20:39 |
| mordred | that's where I expect the weirdest debugging to come | 20:39 |
| *** fawadkhaliq has quit IRC | 20:40 | |
| *** mylu has quit IRC | 20:40 | |
| *** kgiusti has quit IRC | 20:40 | |
| *** fawadkhaliq has joined #openstack-infra | 20:40 | |
| mordred | pabelanger: we should investigate the as-user path | 20:40 |
| mordred | pabelanger: because it might be that we have something that's not working right for that | 20:41 |
| pabelanger | mordred: ack | 20:41 |
| *** hamzy has quit IRC | 20:41 | |
| *** heyongli has quit IRC | 20:42 | |
| *** heyongli has joined #openstack-infra | 20:43 | |
| *** mylu has joined #openstack-infra | 20:46 | |
| *** bdemers has quit IRC | 20:48 | |
| imcsk8 | hello i want to add the job that publishes the packstack tarball to tarballs.openstack.org, for what i've checked i think i just need to add packstack-branch-tarball to the post pipeline in https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L8600 right? | 20:48 |
| AJaeger | imcsk8: yes. But check jenkins/scripts/run-tarball.sh and run it locally to see that you can generate tarballs | 20:50 |
| *** fawadkhaliq has quit IRC | 20:50 | |
| imcsk8 | AJaeger: thanks i will | 20:50 |
| *** fawadkhaliq has joined #openstack-infra | 20:51 | |
| *** ddieterly is now known as ddieterly[away] | 20:52 | |
| *** jaypipes has quit IRC | 20:52 | |
| fungi | pabelanger: sudo su - | 20:52 |
| *** vgridnev has joined #openstack-infra | 20:52 | |
| *** heyongli has quit IRC | 20:52 | |
| *** e0ne has quit IRC | 20:53 | |
| *** heyongli has joined #openstack-infra | 20:53 | |
| openstackgerrit | Chris Dent proposed openstack-infra/project-config: Add a microversion-parse project https://review.openstack.org/296046 | 20:53 |
| *** asettle has joined #openstack-infra | 20:53 | |
| hashar | mordred: (re irc bot maintainer) good to know you can grant root on specific VMs. Guess that let you scale the infra team easily this way ;-} | 20:53 |
| mordred | hashar: yah - we had to expand past all or nothing root | 20:53 |
| hashar | zaro: sorry was away. Most fetches are done from Gerrit, but when it is too slow folks uses the Github mirror | 20:53 |
| *** jamesmcarthur has quit IRC | 20:54 | |
| fungi | hashar: sounds like similar reasons that we deployed separate git server infrastructure for gerrit to push into | 20:55 |
| *** salv-orlando has joined #openstack-infra | 20:56 | |
| fungi | people shouldn't ever actually need to fetch anything directly from our gerrit since we replicate all refs (even just proposed changes) | 20:56 |
| fungi | though right now we don't have the gerrit configuration tweaked to provide the urls correctly for that | 20:56 |
| fungi | so gerrit's "download" links still go to review.o.o | 20:57 |
| fungi | er, download commands | 20:57 |
| *** |-paul-| has joined #openstack-infra | 20:57 | |
| zaro | fungi: i think it's just a gerrit config change to make that happen. | 20:58 |
| *** Sukhdev has joined #openstack-infra | 20:59 | |
| *** nadya has joined #openstack-infra | 20:59 | |
| imcsk8 | AJaeger: if i want the tarball to be created after i tag a release should use the release pipeline? | 20:59 |
| hashar | zaro: this is what it does for our 10+ years old MediaWiki http://paste.openstack.org/show/491510/ | 21:00 |
| AJaeger | jhesketh: could you review https://review.openstack.org/271128 again, please? I commented on your -1 | 21:00 |
| *** thorst has quit IRC | 21:00 | |
| AJaeger | imcsk8: yes | 21:00 |
| imcsk8 | AJaeger: thanks! | 21:00 |
| *** thorst has joined #openstack-infra | 21:00 | |
| fungi | zaro: agreed, i think the gitweb links are a little harder since the browse url formatting for cgit is a bit different, but the fetch/pull/cherry-pick commands should be easy to adjust | 21:01 |
| *** vgridnev has quit IRC | 21:01 | |
| *** IlyaG has quit IRC | 21:01 | |
| *** jamesmcarthur has joined #openstack-infra | 21:01 | |
| fungi | imcsk8: keep in mind that the template for the post pipeline is {name}-branch-tarball while the one you likely want in the pre-release/release pipelines is just {name}-tarball | 21:02 |
| *** EricGonczer_ has quit IRC | 21:02 | |
| *** Mmike has quit IRC | 21:02 | |
| *** Mmike has joined #openstack-infra | 21:02 | |
| fungi | the branch tarballs get replaced on every new commit, but the release tarballs are versioned and kept indefinitelyt | 21:02 |
| fungi | indefinitely | 21:02 |
| *** heyongli has quit IRC | 21:03 | |
| *** heyongli has joined #openstack-infra | 21:03 | |
| imcsk8 | fungi: oh yeah i've noticed that thanks! | 21:03 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Fix on Email Verification https://review.openstack.org/296051 | 21:03 |
| *** thorst has quit IRC | 21:05 | |
| *** mylu has quit IRC | 21:05 | |
| *** mylu has joined #openstack-infra | 21:06 | |
| imcsk8 | just for curiosity sake, which is the script that actually performs the tarballl upload to tarballs.openstack.org | 21:08 |
| AJaeger | imcsk8: it's a publisher | 21:08 |
| AJaeger | imcsk8: see jenkins/jobs/python-jobs.yaml for the full job definition | 21:08 |
| *** nadya has quit IRC | 21:09 | |
| AJaeger | and the publisher is defined in jenkins/jobs/macros.yaml | 21:09 |
| pabelanger | infra-root: I fear I have made ansible wonky on puppetmaster.o.o, I see some defunct processes | 21:09 |
| imcsk8 | AJaeger: thanks! | 21:10 |
| *** IlyaG has joined #openstack-infra | 21:10 | |
| *** jcoufal has quit IRC | 21:10 | |
| pabelanger | I recreated centos-7 wheel-mirror again, so that's I've done in the last 20mins | 21:10 |
| *** regXboi has quit IRC | 21:11 | |
| mordred | imcsk8: jenkins/jobs/macros.yaml - look for 'name: tarball' in a - publisher section in the openstack-infra/project-config rep | 21:11 |
| mordred | repo | 21:11 |
| imcsk8 | mordred: thanks! | 21:11 |
| *** dims has joined #openstack-infra | 21:11 | |
| *** _amrith_ is now known as amrith | 21:11 | |
| openstackgerrit | Thomas Herve proposed openstack-dev/pbr: Fix wsgiref script use with oslo.config https://review.openstack.org/296061 | 21:11 |
| *** nmagnezi has quit IRC | 21:12 | |
| openstackgerrit | Merged openstack-infra/project-config: Create bareon-allocator repository https://review.openstack.org/292765 | 21:12 |
| *** ddieterly[away] is now known as ddieterly | 21:13 | |
| *** heyongli has quit IRC | 21:13 | |
| *** heyongli has joined #openstack-infra | 21:13 | |
| imcsk8 | AJaeger, mordred i understood it thanks a lot guys! | 21:13 |
| *** vgridnev has joined #openstack-infra | 21:14 | |
| * AJaeger waves good bye | 21:14 | |
| ddieterly | what's the process for getting project-config changes workflow approved? | 21:14 |
| ddieterly | we have a blocker issue in our gates and need https://review.openstack.org/#/c/291430/ merged to get the gates unstuck | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Stop updating the board in-place when moving lanes https://review.openstack.org/291786 | 21:15 |
| ddieterly | mordred is there a process for getting changes to project-config approved/merged? | 21:17 |
| fungi | ddieterly: what introduced the regression? | 21:17 |
| fungi | ddieterly: or did that job get added in a broken state without being experimental or non-voting first to make sure it worked? | 21:18 |
| ddieterly | not a regression, we need changes to the gate to get the tempest tests working, but now that tempest tests are in the project repos, the gate won't pass | 21:18 |
| ddieterly | chicken/egg | 21:18 |
| ddieterly | maybe there is a better series of steps to get gates changed along with the accompanying changes to the project repos | 21:19 |
| *** Sukhdev has quit IRC | 21:19 | |
| ddieterly | in this case somebody had already made the gates voting without having the tempest tests in place | 21:19 |
| ddieterly | fungi yea, the job was there and made was changed to voting before the tempest tests were actually in the project | 21:20 |
| ddieterly | fungi the ptl did not want to change the job back to non-voting | 21:21 |
| ddieterly | fungi is there a process for these changes? is there someone who gets poked? | 21:21 |
| openstackgerrit | Merged openstack-infra/openstackid: Fix on Email Verification https://review.openstack.org/296051 | 21:22 |
| *** thorst has joined #openstack-infra | 21:22 | |
| fungi | ddieterly: we have a team of something like 15 people with core review permissions on them, but the change volume is also substantial so 2 days is not a long time for them to get around to reviewing normally but if it's urgent then giving us a heads up in here can sometimes speed up the process | 21:22 |
| ddieterly | fungi ok, i see | 21:23 |
| ddieterly | fungi i see that you approved the change | 21:23 |
| ddieterly | fungi thanks! | 21:23 |
| fungi | i did indeed | 21:23 |
| fungi | yw | 21:23 |
| *** heyongli has quit IRC | 21:23 | |
| *** aeng has joined #openstack-infra | 21:23 | |
| *** heyongli has joined #openstack-infra | 21:23 | |
| pabelanger | okay, so something is up with ansible on puppetmaster.o.o. been stuck for about 45mins on remote_puppet_else. Is it safe to kill those processes or just wait for timeout | 21:25 |
| *** EricGonczer_ has joined #openstack-infra | 21:25 | |
| *** vgridnev has quit IRC | 21:25 | |
| *** mgkwill has quit IRC | 21:26 | |
| *** thorst has quit IRC | 21:26 | |
| *** mgkwill has joined #openstack-infra | 21:26 | |
| openstackgerrit | Merged openstack-infra/shade: Fix race condition in deleting volumes https://review.openstack.org/295821 | 21:27 |
| *** cdent has quit IRC | 21:27 | |
| mordred | WOOT! | 21:27 |
| mordred | Shrews: I think we're ready for the shade release now | 21:28 |
| *** sputnik13 has quit IRC | 21:28 | |
| *** keedya has quit IRC | 21:28 | |
| *** sputnik13 has joined #openstack-infra | 21:28 | |
| openstackgerrit | Merged openstack-infra/project-config: Modify gate-freezer-api-devstack-dsvm job https://review.openstack.org/291430 | 21:30 |
| *** vgridnev has joined #openstack-infra | 21:30 | |
| *** notnownikki has quit IRC | 21:31 | |
| *** mylu has quit IRC | 21:32 | |
| * rcarrillocruz looks forwards to a shade release to push the ansible PRs depending on latest changes | 21:33 | |
| *** heyongli has quit IRC | 21:33 | |
| *** heyongli has joined #openstack-infra | 21:34 | |
| *** achanda has joined #openstack-infra | 21:34 | |
| wznoinsk | hi | 21:34 |
| pabelanger | okay, I've manged to get ansible working again by killing the hung SSH process | 21:35 |
| pabelanger | which was connecting to my stopped centos-7 instance | 21:35 |
| pabelanger | wheel-mirror-centos-7 | 21:36 |
| *** jsavak has quit IRC | 21:36 | |
| *** netsin has quit IRC | 21:36 | |
| *** vgridnev has quit IRC | 21:37 | |
| fungi | pabelanger: oh, yep the timeout on it is longish | 21:37 |
| fungi | noticed the same thing when we shutdown another server we were retiring recently | 21:38 |
| pabelanger | fungi: Ya, it was pushing 60mins before I killed the process manually | 21:38 |
| *** ybathia has quit IRC | 21:38 | |
| fungi | i think that actually may be the timeout | 21:38 |
| craige | o/ | 21:38 |
| wznoinsk | could someone from infra tell me what .htaccess defs they have on logs.openstack.org for in this case testr_results.html.gz, my config doesn't seem to work and don't know exactly why | 21:40 |
| *** achanda has quit IRC | 21:40 | |
| clarkb | wznoinsk: no .htaccess should all be in the vhost | 21:41 |
| *** daemontool has joined #openstack-infra | 21:41 | |
| wznoinsk | could you share what you ahve clarkb ? it's only vps I have, no access to httpd | 21:41 |
| *** tongli has quit IRC | 21:42 | |
| wznoinsk | clarkb, my part of the story: | 21:42 |
| wznoinsk | <FilesMatch "\.html\.gz$"> | 21:42 |
| wznoinsk | ForceType text/html | 21:42 |
| wznoinsk | AddEncoding gzip .gz | 21:42 |
| wznoinsk | </FilesMatch> | 21:42 |
| wznoinsk | and I see the file doesn't always load in full - http://intel-openstack-ci-logs.ovh/zuul/38/295938/1/silent-nfv/3rdparty-nfv-ci/141c5c7/testr_results.html.gz | 21:42 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/templates/logs.vhost.erb | 21:43 |
| *** catherineD has left #openstack-infra | 21:43 | |
| *** vgridnev has joined #openstack-infra | 21:43 | |
| *** sc68cal has quit IRC | 21:44 | |
| *** vgridnev has quit IRC | 21:44 | |
| *** heyongli has quit IRC | 21:44 | |
| *** heyongli has joined #openstack-infra | 21:44 | |
| openstackgerrit | Ian Cordasco proposed openstack-dev/pbr: Include wsgi_scripts in generated wheels https://review.openstack.org/280270 | 21:44 |
| *** ddieterly is now known as ddieterly[away] | 21:47 | |
| *** openstackgerrit has quit IRC | 21:48 | |
| *** openstackgerrit has joined #openstack-infra | 21:49 | |
| *** |-paul-| has quit IRC | 21:50 | |
| *** e0ne has joined #openstack-infra | 21:50 | |
| openstackgerrit | Ivan Chavero proposed openstack-infra/project-config: Add tarball creation to packstack project https://review.openstack.org/296084 | 21:53 |
| *** mylu has joined #openstack-infra | 21:53 | |
| *** heyongli has quit IRC | 21:54 | |
| *** Sukhdev has joined #openstack-infra | 21:54 | |
| *** heyongli has joined #openstack-infra | 21:54 | |
| *** ddieterly[away] is now known as ddieterly | 21:54 | |
| *** sputnik13 has quit IRC | 21:55 | |
| *** mriedem has quit IRC | 21:58 | |
| *** ihrachys has quit IRC | 21:59 | |
| pabelanger | fungi: why would ansible appear to be hung on review.o.o? | 22:00 |
| *** sputnik13 has joined #openstack-infra | 22:00 | |
| fungi | pabelanger: have you looked on review.o.o in syslog to see if it's in the middle of running manage-projects? that can take a while | 22:01 |
| pabelanger | fungi: Ya, looks to be that | 22:01 |
| pabelanger | fungi: thanks | 22:01 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:02 | |
| *** dimtruck is now known as zz_dimtruck | 22:03 | |
| *** heyongli has quit IRC | 22:04 | |
| *** EricGonczer_ has quit IRC | 22:04 | |
| *** heyongli has joined #openstack-infra | 22:05 | |
| *** rkukura has quit IRC | 22:05 | |
| *** eharney has quit IRC | 22:06 | |
| ddieterly | fungi how long after project-config is merged does it take for the changes to show up in the gate jobs? | 22:06 |
| *** baoli has joined #openstack-infra | 22:07 | |
| *** annegentle has quit IRC | 22:08 | |
| *** johnsom has quit IRC | 22:08 | |
| pabelanger | ansible runs every 15mins | 22:08 |
| *** harlowja_at_home has joined #openstack-infra | 22:08 | |
| pabelanger | so, you should be able to monitor puppetdb.o.o to see the results of jenkins | 22:08 |
| anteaya | hashar: umm, thanks for sharing that scenario, we tend to be rather stingy with root access to things around here, so not sure how your suggestion would play out | 22:09 |
| *** mylu has quit IRC | 22:09 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Fix on missing refresh token when consent params had more than 2 values https://review.openstack.org/296087 | 22:10 |
| anteaya | hashar: I'm thinking I will draft an email to the dev list tomorrow on an etherpad and try to highlight some of the items mentioned in today's meeting opening up for further discussion, perhaps you can add your suggestion as a post so others can consider it? | 22:10 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Use the mirror server for the centos image and git repos https://review.openstack.org/285257 | 22:10 |
| *** baoli has quit IRC | 22:11 | |
| anteaya | so here is zaro's patch to adjust logging levels for gearman plugin: https://review.openstack.org/#/c/82627/ fungi yolanda | 22:12 |
| *** hogepodge has quit IRC | 22:12 | |
| jeblair | what has changed, and why only on those jenkins hosts? | 22:13 |
| *** mylu has joined #openstack-infra | 22:14 | |
| jeblair | anteaya, zaro, fungi, yolanda: ^ | 22:14 |
| *** darrenc is now known as darrenc_afk | 22:14 | |
| jeblair | i think we should understand that before we merge that patch | 22:14 |
| fungi | is it only some jenkins masters which are filling up? | 22:14 |
| fungi | i wasn't following closely | 22:14 |
| *** heyongli has quit IRC | 22:14 | |
| jeblair | that was what i was lead to believe. | 22:15 |
| *** heyongli has joined #openstack-infra | 22:15 | |
| anteaya | fungi: yes | 22:15 |
| fungi | i saw mention of two which ran out of space completely on their rootfs, but wasn't sure if the others were just soon to follow or completely unaffected | 22:15 |
| anteaya | only 03 and 05 were filled | 22:15 |
| anteaya | 01 and 02 are almost empty on disk | 22:16 |
| *** rkukura has joined #openstack-infra | 22:16 | |
| anteaya | 04 06 and 07 are medium | 22:16 |
| jeblair | what was done to reduce the space used? | 22:16 |
| anteaya | jeblair: and yes, I agree we should understand why there is a difference | 22:16 |
| anteaya | that was what cacti showed me when I looked a disk space | 22:16 |
| anteaya | 01 and 02 have always used minimal disk space as far back as I can see in cacti | 22:17 |
| *** annegentle has joined #openstack-infra | 22:17 | |
| *** aysyd has quit IRC | 22:17 | |
| jeblair | anteaya: 01 and 02 simply have larger disks | 22:17 |
| anteaya | ah there we go | 22:18 |
| fungi | i can take a look at graphs once i'm done eating dinner, but am mostly interested in whether 03 and 05 filled up "suddenly" or showed steady growth in filesystem usage prior to the upgrade | 22:18 |
| anteaya | 1T comapared to 30G on the rest | 22:18 |
| lifeless | clarkb: mordred: ping- https://review.openstack.org/280270 | 22:19 |
| anteaya | fungi: started increased between the 18 and 19th with a large jump on the 21st: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1437&rra_id=all | 22:19 |
| mordred | lifeless: funny, I was just looking at that | 22:19 |
| lifeless | mordred: good :) | 22:20 |
| *** e0ne has quit IRC | 22:21 | |
| lifeless | mordred: don't delay, +2 today | 22:21 |
| *** achanda has joined #openstack-infra | 22:21 | |
| *** bpokorny has quit IRC | 22:22 | |
| *** samuelBartel has quit IRC | 22:23 | |
| *** mylu has quit IRC | 22:23 | |
| *** zeih has quit IRC | 22:23 | |
| *** heyongli has quit IRC | 22:25 | |
| *** heyongli has joined #openstack-infra | 22:25 | |
| *** tonytan_brb has quit IRC | 22:28 | |
| *** IlyaG has quit IRC | 22:29 | |
| *** rkukura has quit IRC | 22:30 | |
| *** Sukhdev has quit IRC | 22:31 | |
| *** gordc has quit IRC | 22:32 | |
| *** baoli has joined #openstack-infra | 22:32 | |
| *** mylu has joined #openstack-infra | 22:33 | |
| *** rkukura has joined #openstack-infra | 22:33 | |
| *** Sukhdev has joined #openstack-infra | 22:33 | |
| *** zeih has joined #openstack-infra | 22:33 | |
| *** baoli_ has joined #openstack-infra | 22:33 | |
| *** jamesmcarthur has quit IRC | 22:34 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Support py34 functional tests for Oslo.Messaging https://review.openstack.org/295531 | 22:35 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Switch voting to false for oslo.messaging+dsvm jobs https://review.openstack.org/296093 | 22:35 |
| *** heyongli has quit IRC | 22:35 | |
| *** amitgandhinz has quit IRC | 22:35 | |
| *** darrenc_afk is now known as darrenc | 22:35 | |
| *** heyongli has joined #openstack-infra | 22:35 | |
| dims | anteaya : thanks for the feedback. ^^ | 22:35 |
| *** baoli has quit IRC | 22:37 | |
| *** Sukhdev has quit IRC | 22:37 | |
| *** hashar has quit IRC | 22:37 | |
| *** gildub has joined #openstack-infra | 22:38 | |
| *** IlyaG has joined #openstack-infra | 22:38 | |
| *** annegentle has quit IRC | 22:38 | |
| *** ddieterly is now known as ddieterly[away] | 22:39 | |
| *** ddieterly[away] has quit IRC | 22:39 | |
| *** hogepodge has joined #openstack-infra | 22:39 | |
| *** ociuhandu has quit IRC | 22:40 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Fix on missing refresh token when consent params had more than 2 values Fixed unit tests Change-Id: I8800c3d768794bdd1d99c558225665e9e9ee13a0 https://review.openstack.org/296087 | 22:41 |
| *** achanda has quit IRC | 22:43 | |
| *** ayoung has quit IRC | 22:43 | |
| *** Sukhdev has joined #openstack-infra | 22:44 | |
| *** heyongli has quit IRC | 22:45 | |
| *** heyongli has joined #openstack-infra | 22:45 | |
| *** smarcet has quit IRC | 22:46 | |
| anteaya | dims: thank you! | 22:47 |
| * anteaya reviews | 22:47 | |
| openstackgerrit | Merged openstack-infra/openstackid: Fix on missing refresh token when consent params had more than 2 values Fixed unit tests Change-Id: I8800c3d768794bdd1d99c558225665e9e9ee13a0 https://review.openstack.org/296087 | 22:49 |
| *** knikolla has quit IRC | 22:49 | |
| *** rlandy has quit IRC | 22:49 | |
| anteaya | dims: +2 on both, thank you | 22:50 |
| anteaya | dims: AJaeger is offline for the night but I'm betting will include these in his morning reviews unless some other project-config core reviewer beats him to them | 22:50 |
| anteaya | thank you | 22:50 |
| anteaya | I'm going to offline for a bit myself | 22:51 |
| *** bpokorny has joined #openstack-infra | 22:54 | |
| *** austin81 has left #openstack-infra | 22:55 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Fix python3 devel for CentOS on wheel mirror slave https://review.openstack.org/296097 | 22:55 |
| *** heyongli has quit IRC | 22:55 | |
| *** heyongli has joined #openstack-infra | 22:56 | |
| *** rockyg has quit IRC | 22:57 | |
| pabelanger | clarkb: fungi: ianw: wheel-mirror-centos-7-amd64 now online! | 22:57 |
| pabelanger | only issue I found was above ^ | 22:58 |
| openstackgerrit | Sachi King proposed openstack-infra/zuul: Support post jobs by supporting rev checkout https://review.openstack.org/293194 | 22:58 |
| pabelanger | clarkb: fungi: ianw: So, I think we can try 290272 now | 22:59 |
| *** annegentle has joined #openstack-infra | 22:59 | |
| *** ociuhandu has joined #openstack-infra | 22:59 | |
| *** baoli_ has quit IRC | 23:00 | |
| *** apoorvad has quit IRC | 23:01 | |
| *** baoli has joined #openstack-infra | 23:01 | |
| *** zz_dimtruck is now known as dimtruck | 23:02 | |
| *** annegentle has quit IRC | 23:04 | |
| *** heyongli has quit IRC | 23:06 | |
| *** heyongli has joined #openstack-infra | 23:06 | |
| dims | anteaya : thanks | 23:06 |
| *** IlyaG has quit IRC | 23:06 | |
| *** apoorvad has joined #openstack-infra | 23:06 | |
| *** zeih has quit IRC | 23:06 | |
| *** IlyaG has joined #openstack-infra | 23:07 | |
| *** annegentle has joined #openstack-infra | 23:09 | |
| *** annegentle has quit IRC | 23:10 | |
| *** dims_ has joined #openstack-infra | 23:12 | |
| *** dingyichen has joined #openstack-infra | 23:14 | |
| *** dims has quit IRC | 23:15 | |
| *** dims has joined #openstack-infra | 23:15 | |
| *** tonytan4ever has joined #openstack-infra | 23:15 | |
| *** heyongli has quit IRC | 23:16 | |
| *** baoli has quit IRC | 23:16 | |
| *** heyongli has joined #openstack-infra | 23:16 | |
| *** rbrndt has quit IRC | 23:17 | |
| *** dims_ has quit IRC | 23:18 | |
| *** dims_ has joined #openstack-infra | 23:19 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add centos7 to think_slave.pp https://review.openstack.org/296097 | 23:20 |
| wznoinsk | clarkb, is .htaccess any problematic hence all in vhost or it's just convenience thing ? | 23:21 |
| clarkb | wznoinsk: performance is better inthe vhost iircso we prefer it | 23:21 |
| *** achanda has joined #openstack-infra | 23:22 | |
| *** dims has quit IRC | 23:22 | |
| wznoinsk | clarkb, ok, cheers | 23:22 |
| *** tonytan_brb has joined #openstack-infra | 23:22 | |
| *** Jeffrey4l has quit IRC | 23:23 | |
| *** dims has joined #openstack-infra | 23:23 | |
| *** mylu has quit IRC | 23:24 | |
| *** tonytan4ever has quit IRC | 23:25 | |
| *** dims_ has quit IRC | 23:25 | |
| *** mylu has joined #openstack-infra | 23:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Fix centos7 issue with wheel_mirror.pp https://review.openstack.org/296097 | 23:26 |
| *** heyongli has quit IRC | 23:26 | |
| *** Qiming has joined #openstack-infra | 23:26 | |
| *** heyongli has joined #openstack-infra | 23:27 | |
| *** tonytan4ever has joined #openstack-infra | 23:28 | |
| *** mylu has quit IRC | 23:28 | |
| fungi | also there are things you can't set in .htaccess for various reasons | 23:30 |
| fungi | and some variables you don't have access to | 23:30 |
| *** IlyaG has quit IRC | 23:30 | |
| *** tonytan4_ has joined #openstack-infra | 23:31 | |
| *** tonytan_brb has quit IRC | 23:31 | |
| *** MarkAtwood has quit IRC | 23:32 | |
| *** tonytan4ever has quit IRC | 23:34 | |
| *** mylu has joined #openstack-infra | 23:35 | |
| *** amitgandhinz has joined #openstack-infra | 23:36 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add note about running keystone on 443 https://review.openstack.org/201667 | 23:36 |
| *** kzaitsev_mb has quit IRC | 23:36 | |
| *** Sukhdev has quit IRC | 23:36 | |
| *** heyongli has quit IRC | 23:36 | |
| *** heyongli has joined #openstack-infra | 23:37 | |
| *** dimtruck is now known as zz_dimtruck | 23:38 | |
| *** EricGonczer_ has joined #openstack-infra | 23:39 | |
| clarkb | pabelanger: so you got it all done? cool | 23:39 |
| *** knikolla has joined #openstack-infra | 23:39 | |
| pabelanger | clarkb: Yup, only 2 issue around mysql-devel and python-devel. Patch up to system-config to fix that | 23:40 |
| *** mylu has quit IRC | 23:40 | |
| pabelanger | clarkb: just talking with ianw about https://review.openstack.org/#/c/290272/ now | 23:40 |
| pabelanger | how to best move forward with a build | 23:40 |
| clarkb | pabelanger: https://apps.fedoraproject.org/packages/python3-devel | 23:40 |
| clarkb | pabelanger: I think ^ means oyu need a richer case statement around python3 dev packages | 23:41 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Add centos wheel build as experimental https://review.openstack.org/296115 | 23:41 |
| *** mylu has joined #openstack-infra | 23:41 | |
| pabelanger | clarkb: right, but doesn't exist in epel7 but python34-devel does | 23:41 |
| clarkb | pabelanger: a nested case under red hat, one for rhel/centos and one for fedora? | 23:41 |
| pabelanger | https://apps.fedoraproject.org/packages/python34 | 23:42 |
| pabelanger | clarkb: maybe | 23:42 |
| pabelanger | clarkb: will look at thick_slave.pp again | 23:42 |
| pabelanger | clarkb: Oh, actually, this is just for wheel_mirror_slave.pp, so we are fine there | 23:43 |
| clarkb | pabelanger: until we want wheels for fedora | 23:43 |
| *** kzaitsev_mb has joined #openstack-infra | 23:43 | |
| clarkb | but ya maybe just worry about that when its time | 23:43 |
| *** markvoelker has quit IRC | 23:44 | |
| pabelanger | clarkb: Ya, I thought we didn't want that because of the churn of fedora releases? | 23:44 |
| pabelanger | either way, something to keep in mind | 23:44 |
| clarkb | ya thats a good point and f22 to f23 did a massive abi change I think | 23:44 |
| clarkb | maybe it was f21 to f22 | 23:44 |
| openstackgerrit | Zhijiang Hu proposed openstack-infra/project-config: [new project] Create a openstack related project "daisycloud-core" https://review.openstack.org/296117 | 23:44 |
| fungi | mordred: previously and still, when running launch-node.py i'm seeing this error a bunch of times at the end: /etc/ansible/hosts/openstack:43: Expected key=value host variable assignment, got: argparse | 23:46 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release https://review.openstack.org/293155 | 23:46 |
| *** heyongli has quit IRC | 23:47 | |
| *** markvoelker has joined #openstack-infra | 23:47 | |
| clarkb | hopefully my puppet will stop being terrible and ^ will actually pass tests | 23:47 |
| *** heyongli has joined #openstack-infra | 23:47 | |
| pabelanger | clarkb: ianw: if we land 296115 in the next hour, I can still help debug this evening. Other wise, will have to finish off testing in the morning | 23:48 |
| *** fawadkhaliq has quit IRC | 23:48 | |
| openstackgerrit | Merged openstack-dev/pbr: Include wsgi_scripts in generated wheels https://review.openstack.org/280270 | 23:49 |
| *** fawadkhaliq has joined #openstack-infra | 23:49 | |
| fungi | mordred: also it looks to me like it didn't actually apply any server-specific puppet when deploying | 23:49 |
| *** amitgandhinz has quit IRC | 23:49 | |
| ianw | pabelanger: it will probably take zuul a while to reload it anyway... | 23:49 |
| clarkb | pabelanger: ianw I don't know that that change is "safe" you could write a project-config change to updat ethe script and check experimental it to expose the kerberos stuff | 23:49 |
| fungi | i may still need to roll back to older launch-node.py for the moment | 23:50 |
| mordred | fungi: mind if I drive for a sec? | 23:50 |
| pabelanger | clarkb: how did we test ubuntu-trusty wheel builds BTW? | 23:50 |
| mordred | I'd really like to not roll back to older versions of things | 23:50 |
| fungi | mordred: feel free. i can blow this one away, deploy another in a root screen session if you want to -x? | 23:50 |
| clarkb | pabelanger: just add the job to periodic and you can trigger it out of band | 23:51 |
| mordred | fungi: you were running as root that time? | 23:51 |
| *** tqtran has quit IRC | 23:52 | |
| pabelanger | clarkb: okay, so I think we need to land https://review.openstack.org/#/c/290272/ then | 23:52 |
| pabelanger | or a modified version | 23:52 |
| fungi | mordred: yeah | 23:53 |
| fungi | mordred: should i not have been? i was under the impression new ansibled launcher broke being able to run as non-root | 23:53 |
| mordred | yes. just making sure | 23:54 |
| fungi | mordred: anyway i have a root screen session going if you want to join, ready to pull the trigger after you have a chance to see the variables i'm exporting | 23:54 |
| *** kzaitsev_mb has quit IRC | 23:54 | |
| clarkb | pabelanger: also nothing will use this until we update the ready scripts for the nodepool nodes so its safe for it to be broken for a while | 23:54 |
| *** baoli has joined #openstack-infra | 23:54 | |
| clarkb | pabelanger: approved, if you want it to run I think you can just trigger it manually | 23:55 |
| *** rfolco has quit IRC | 23:55 | |
| pabelanger | clarkb: ok | 23:55 |
| clarkb | if you don't need any of the zuul vars just hitting the jenkins button to run now should work | 23:55 |
| clarkb | if you do need the zuul vars I think you can enqueue stuff with zuul-enqueue in periodic queues now | 23:56 |
| fungi | mordred: the version in /opt isn't going to work because of no executable permission on the script | 23:56 |
| fungi | old version of launch-node.py was +x but new version which replaced it is not | 23:57 |
| mordred | has anyone made a patch for that? | 23:57 |
| *** ajmiller has quit IRC | 23:57 | |
| pabelanger | clarkb: Ya, I don't think we need any zuul vars | 23:57 |
| *** heyongli has quit IRC | 23:57 | |
| fungi | mordred: not yet, i've just been chmodding while we worked out the more serious issues | 23:57 |
| *** heyongli has joined #openstack-infra | 23:57 | |
| clarkb | pabelanger: made a patch to +x it | 23:57 |
| clarkb | fungi: mordred ^ | 23:58 |
| fungi | presumably others have been running it in a manner inconsistent with the readme (prepending the interpreter?) | 23:58 |
| fungi | oh, excellent | 23:58 |
| fungi | i missed that one whir by | 23:58 |
| pabelanger | Ya, I need to rebase it | 23:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Give 0775 to launch-node.py https://review.openstack.org/295959 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!