| ianw | clarkb: we need to be like the high-frequency traders and make sure we have a dedicated pipe into github at the same colo | 00:00 |
|---|---|---|
| *** eernst has quit IRC | 00:01 | |
| mordred | ianw: or, just continue to work on reducing how much people care about the replication in the first place | 00:01 |
| *** claudiub has quit IRC | 00:01 | |
| *** aeng has quit IRC | 00:04 | |
| *** pahuang has quit IRC | 00:05 | |
| clarkb | into the M's | 00:06 |
| clarkb | nova-spec should be soon | 00:06 |
| ianw | how lame is it that the most exciting part of my day is watching these logs and waiting to see if it's fixed :) | 00:07 |
| notmyname | clarkb: I want to start working towards the 3rd party testing I was talking about. either in the project referenced in https://review.openstack.org/548909 or a different project that's a little simpler to set up | 00:08 |
| *** jamesmcarthur has joined #openstack-infra | 00:09 | |
| notmyname | clarkb: this is the other repo: https://github.com/swiftstack/swift-s3-sync. the readme describes testing. it "just" requires installing docker/docker-compose and running a command. I'd love to have those tests run non-voting on every swift patch proposed | 00:10 |
| persia | Do I understand correctly that for those, it is possible to define and run arbitrary jobs, but not configure roles, nodesets, etc.? | 00:10 |
| *** smarunich has joined #openstack-infra | 00:10 | |
| notmyname | clarkb: so if my current understanding is correct, I need to describe the test setup and call it from some zuul job definition in the project config repo | 00:11 |
| notmyname | (I don't really know how to do that yet, but if that's the thing to do, I can start researching it) | 00:11 |
| persia | I'm not authoritative, but I think you'd *call* the tests from something in the swift repo. Setup is more complicated (and I don't really understand where that needs to live). | 00:12 |
| ianw | woo hoo https://github.com/openstack/nova-specs/commits/master back in sync | 00:13 |
| *** jamesmcarthur has quit IRC | 00:13 | |
| clarkb | notmyname: you can do it from swift not just project-config | 00:13 |
| notmyname | persia: that doesn't feel right, since the 3rd party repo is built on swift, not the other way around | 00:13 |
| clarkb | notmyname: it just has to be infra hosted currently | 00:13 |
| clarkb | notmyname: persia correct the include: [] a few lines above notmyname's change means we can't load that from swift-s3-sync | 00:14 |
| persia | notmyname: My logic is that if you want something to run on every change to swift, the job belongs in the swift check list, which I'd expect to find in swift:.zuul.d/jobs.yaml | 00:14 |
| clarkb | or swiftstack/ProxyFS | 00:14 |
| clarkb | persia: ya I think putting it in swift would be the right location | 00:14 |
| *** slaweq has joined #openstack-infra | 00:14 | |
| notmyname | ignoring the "protect the infra resources" part, I think the job definition would belong in the 3rd party repo, not swift. and zuul detects the proposed change to swift and calls the 3rd party test | 00:15 |
| persia | The part I don't understand is that presumably one wants to do something to get the tests out of the foreign repo, to run them. For example, clone master of the foreign and run something. Is it safe to clone from outside infra-hosted stuff as dependencies for a job? | 00:15 |
| notmyname | in this case, the job definition would be in project_config for the reasons stated earlier | 00:16 |
| persia | notmyname: Be careful to semantically distinguish between "zuul job definition being run in the check queue" and "some arbitrary set of code that is run that returns something". | 00:16 |
| clarkb | persia: it should be safe from a security standpoint since we isolate the job runners and they are single use | 00:16 |
| clarkb | it may be slightly less reliable due to github outages | 00:16 |
| persia | clarkb: That makes it easy then. | 00:16 |
| notmyname | the basic problem is "I've got two non-openstack projects hosted on github that are built on swift. I want to make sure that their patches work on latest swift and swift's patches don't break them" | 00:17 |
| persia | notmyname: In that case, yeah, just create a job in the swift repo that runs on check and clones something from github and then runs something in the clone. | 00:17 |
| persia | Make it non-voting so that if github clone fails, CI doesn't fail (or be ready to recheck) | 00:17 |
| notmyname | no, that doesn't make sense to me. sounds semantically the same as the XFS project adding swift's tests because swift uses XFS. that's backwards | 00:18 |
| notmyname | *adding swift's tests to the xfs repo | 00:18 |
| *** jamesmcarthur has joined #openstack-infra | 00:19 | |
| persia | notmyname: For the other side, put a .zuul.d/jobs.yaml in the github repo, register with zuul (following the procedure under review), and have the job from there consume the job setup from the swift repo. | 00:19 |
| *** slaweq has quit IRC | 00:19 | |
| persia | notmyname: I thought you wanted A) run external test on every change to swift + B) run external test on every change to external | 00:20 |
| clarkb | ianw: \o/ | 00:20 |
| notmyname | a few months ago, I'd have solved this by setting up jenkins, finding my own test infrastructure, and doing the 3rd party test job integration. but with the new zuul changes, it seems that using openstack's CI infrastructure can really help out (mostly with the "find hardware" problem, but also the "I'm using jenkins" problem) | 00:20 |
| pabelanger | I don't think we'll load jobs from github.com right now | 00:21 |
| pabelanger | it is disabled | 00:21 |
| persia | pabelanger: Yes, but enabling it is described in a proposal under review. | 00:21 |
| notmyname | yeah, I'm walking through the enabling process now for one of the projects | 00:21 |
| pabelanger | persia: I missed that, can you link | 00:21 |
| *** caphrim007_ has joined #openstack-infra | 00:21 | |
| *** aeng has joined #openstack-infra | 00:21 | |
| ianw | i've restarted nb03 ... nodepool-builder was odd state without logging and maybe i accidentally started it by hand at some point. anyway, will monitor | 00:21 |
| *** smarunich has quit IRC | 00:22 | |
| persia | pabelanger: I'm wrong: it merged: https://review.openstack.org/#/c/545077/ | 00:22 |
| *** pahuang has joined #openstack-infra | 00:23 | |
| notmyname | persia: if I understand you correctly, this means after registering the external project, I can create zuul job definitions there. this seems to be counter to what clarkb said earlier | 00:23 |
| *** jamesmcarthur has quit IRC | 00:23 | |
| pabelanger | yah, I think there is still some concern about loading .zuul.yaml from github ATM. But, suspect some point in the future that might change | 00:23 |
| *** gouthamr has joined #openstack-infra | 00:24 | |
| notmyname | are there docs for a .zuul.d directory? swift currently has a .zuul.yaml, but no directory | 00:24 |
| notmyname | TBH, docs for the whole zuul file/folder format and syntax would be helpful | 00:24 |
| *** jamesmcarthur has joined #openstack-infra | 00:25 | |
| *** slaweq has joined #openstack-infra | 00:25 | |
| *** caphrim007 has quit IRC | 00:25 | |
| persia | pabelanger: I think https://review.openstack.org/#/c/549191/ is the one blocking some stuff, but I read that as "allow jobs to be defined, but nothing else". I may have misread. | 00:25 |
| *** jamesmcarthur has quit IRC | 00:25 | |
| pabelanger | notmyname: https://docs.openstack.org/infra/zuul/user/config.html | 00:26 |
| *** jamesmcarthur has joined #openstack-infra | 00:26 | |
| pabelanger | will have docs on zuul.yaml / zuul.d | 00:26 |
| persia | notmyname: I think you want https://docs.openstack.org/infra/zuul/user/config.html#configuration-loading | 00:26 |
| notmyname | pabelanger: thanks | 00:26 |
| clarkb | persia: that change means don't load anything not even jobs | 00:26 |
| clarkb | persia: could be include: [jobs] to get jobs only | 00:27 |
| pabelanger | yah | 00:27 |
| pabelanger | 549191 really just gets the git repos onto executors | 00:27 |
| pabelanger | then jobs in openstack tenant are able to interact with those projects | 00:28 |
| persia | Hrm. Indeed. I thought I saw an earlier publication that just allowed jobs, but it is apparently out of date. Apologies for the confusion. | 00:28 |
| pabelanger | eg: https://review.openstack.org/#/c/549768/21/.zuul.yaml | 00:28 |
| pabelanger | for kubespray | 00:28 |
| clarkb | pabelanger: we can also trigger jobs on their updates | 00:28 |
| pabelanger | clarkb: only if they added openstack github hook thing | 00:29 |
| pabelanger | so, right now we don't see anything from kubernetes-incubator/kubespray | 00:29 |
| clarkb | ya | 00:29 |
| clarkb | but those are the two big things you get out of it without also loading the configs | 00:29 |
| *** slaweq has quit IRC | 00:29 | |
| pabelanger | ya | 00:30 |
| persia | Where would one put the project: definition for an external repo? | 00:30 |
| notmyname | zuul running external jobs in an external project sounds like a travis ci replacement | 00:30 |
| pabelanger | persia: openstack-zuul-jobs would be fine I think | 00:30 |
| persia | So, if one wants to run jobs foo and bar against repo baz on every change in baz (hosted on github), one submits a change to project-config to include the repo, and then one to openstack-zuul-jobs to define and execute the tests? | 00:31 |
| persia | And, if one wants to also run the baz tests on every change in swift, one presumably includes those jobs in the swift project queues? | 00:32 |
| clarkb | ya in this case I think the job can either be defined in swift or in ozj then just add it to the swift queues in swifts .zuul.yaml | 00:33 |
| pabelanger | almost, jobs could be in ozj or specific project (eg: shade) but pipeline configuration needs to be in project-config. As an example, here is ansible/ansible and shade: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n4 | 00:33 |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/shade/tree/.zuul.yaml#n185 | 00:33 |
| notmyname | trying to separate the two different things here.... | 00:33 |
| clarkb | but say we wanted to test pip pull requests to make sure they dn't break us (which next pip release will do btw :( ) we could define that job in ozj and the pipelines in project-config | 00:34 |
| notmyname | for the first one ("one wants to run jobs foo and bar against repo baz on every change in baz (hosted on github), one submits a change to project-config to include the repo, and then one to openstack-zuul-jobs to define and execute the tests") | 00:34 |
| notmyname | are you talking about that? | 00:34 |
| notmyname | or the second one? ("one wants to also run the baz tests on every change in swift, one presumably includes those jobs in the swift project queues | 00:34 |
| clarkb | notmyname: my pip example is for the first case | 00:34 |
| *** slaweq has joined #openstack-infra | 00:35 | |
| notmyname | pabelanger: is that what you were talking about too? | 00:35 |
| *** masuberu has quit IRC | 00:35 | |
| pabelanger | execute the test, needs to be in project-config | 00:35 |
| pabelanger | because only a config-project is able to do that for other projects | 00:36 |
| notmyname | I don't know what a "config project" is | 00:36 |
| persia | To restate, "baz" needs to be defined as a repo in project-config, and pipelines defined in project-config. "foo" and "bar" would go in openstack-zuul-jobs, and be referenced by the pipeline in project-config. "foo" and "bar" might also appear in swift's .zuul.yaml | 00:37 |
| *** masuberu has joined #openstack-infra | 00:37 | |
| notmyname | please stop adding more helpful follow-on info :-) I'm confused enough | 00:37 |
| clarkb | ianw: I think gerrit is done replicating now and no queued nova-specs | 00:37 |
| pabelanger | https://docs.openstack.org/infra/zuul/user/config.html better explains it, but we have 2 types of configuration, trusted (config-project) and untrusted. Certain config stanza is only able to live in a trusted context | 00:37 |
| notmyname | just for the first case | 00:38 |
| clarkb | notmyname: for the first case in todays current situation of not loading configuration from github projects if you want to run tests against pull requests to that github project the job can be defined in any infra hosted project but the pipeline definition that says run that job in check must be in project-config | 00:39 |
| ianw | clarkb: excellent, that's the first time the queue's been empty in quite a bit. i'll keep an eye, but i'm tempted to call this one fixed | 00:39 |
| clarkb | if we were to load the config from the github project then you could define the jobs and the pipeline configs there as well | 00:39 |
| *** slaweq has quit IRC | 00:40 | |
| notmyname | ok. so I need to make another patch to the project-config repo to define tests to be run against every pull request for proxyfs. after https://review.openstack.org/#/c/548909/ lands | 00:40 |
| clarkb | notmyname: if that is what you want to do. Earlier it sounded like you wanted to run tests against swift changes (not github pull requests) | 00:41 |
| *** xinliang has quit IRC | 00:41 | |
| notmyname | that's the other way :-) | 00:41 |
| clarkb | ya | 00:41 |
| notmyname | ok, so that other way | 00:41 |
| notmyname | I want to run proxyfs tests against every proposed patch to swift (a la the old third party testing) | 00:42 |
| notmyname | in this case, openstack-infra is the third party | 00:42 |
| notmyname | running tests defined in project config that pulls code from a github project (proxyfs in this case) | 00:43 |
| notmyname | is that correct? | 00:43 |
| clarkb | yes that is a valid way to do things | 00:43 |
| *** agopi has joined #openstack-infra | 00:44 | |
| clarkb | for that you can define the job in an infra gerrit hosted repo. For this case openstack-zuul-jobs or swift itself are probably the best locations. Then you modify swift's pipeline config (which I think is in swift's .zuul.yaml at this point) to say "run this job for check" | 00:44 |
| notmyname | interesting | 00:44 |
| notmyname | I don't particularly like the idea of putting the job definition of a third party project into swift | 00:45 |
| *** slaweq has joined #openstack-infra | 00:45 | |
| clarkb | in this case I'm not really sure its a job for a third party project | 00:45 |
| clarkb | if it were the other way around and we were testing the github repo on every pull request I would agree | 00:46 |
| *** jamesmcarthur has quit IRC | 00:48 | |
| *** slaweq has quit IRC | 00:50 | |
| notmyname | clarkb: on https://review.openstack.org/#/c/547478/ on feb 23 there was a third party CI test called "swiftstack cluster CI". I think what I'm trying to do, in principle, is move that job definition away from my jenkins box and the hardware used away from my hardware | 00:50 |
| clarkb | yup | 00:50 |
| notmyname | in the proxyfs case, I want to have something similar defined so that proxyfs is tested on a patch to swift | 00:51 |
| clarkb | fwiw you can do that without the github integration as well | 00:51 |
| clarkb | what the github integration should get you in this case is slightly more reliable git repos since zuul will be caching a few more things | 00:51 |
| notmyname | running in -infra infrastructure? | 00:51 |
| clarkb | (so I think its worthwhile) | 00:51 |
| clarkb | ya what the github integration buys you here is simple repo management, if you say a job requires that repo it will be available to your testing. But you can just clone the repo instead in the job | 00:52 |
| *** jamesmcarthur has joined #openstack-infra | 00:53 | |
| *** xinliang has joined #openstack-infra | 00:54 | |
| *** jklare has quit IRC | 00:54 | |
| notmyname | clarkb: pabelanger: persia: thank you for being patient and helping me. | 00:55 |
| clarkb | a lot of testing did (and still does) operate in this way, for example a lot of the puppet testing just clones puppet modules from github and consumes them that way | 00:55 |
| *** slaweq has joined #openstack-infra | 00:55 | |
| notmyname | I'm going to keep going down this road. probably with a slightly simpler repo than proxyfs, but identical in principle | 00:55 |
| persia | The idea of defining the external in zuul and then just requiring it seems cleaner. I hadn't realised we had something that rich available. | 00:56 |
| *** calbers has quit IRC | 00:56 | |
| *** slaweq_ has quit IRC | 00:57 | |
| *** slaweq has quit IRC | 01:00 | |
| *** jamesmcarthur has quit IRC | 01:02 | |
| *** slaweq has joined #openstack-infra | 01:05 | |
| *** germs has joined #openstack-infra | 01:06 | |
| *** germs has quit IRC | 01:06 | |
| *** germs has joined #openstack-infra | 01:06 | |
| *** jklare has joined #openstack-infra | 01:08 | |
| *** gyee has quit IRC | 01:08 | |
| hogepodge | If anyone can take a look at this, it would be cool. We're trying to prep for a major project update tomorrow, and this is a roadblock for pulling in the new upstream code. https://review.openstack.org/#/c/550271/ | 01:10 |
| *** slaweq has quit IRC | 01:10 | |
| *** agopi has quit IRC | 01:10 | |
| *** david-lyle has joined #openstack-infra | 01:10 | |
| *** germs has quit IRC | 01:11 | |
| *** agopi has joined #openstack-infra | 01:15 | |
| *** slaweq has joined #openstack-infra | 01:16 | |
| *** slaweq has quit IRC | 01:20 | |
| *** hongbin has joined #openstack-infra | 01:24 | |
| *** jamesmcarthur has joined #openstack-infra | 01:24 | |
| *** slaweq has joined #openstack-infra | 01:26 | |
| *** lbragstad has quit IRC | 01:27 | |
| *** jamesmca_ has joined #openstack-infra | 01:29 | |
| *** jamesmcarthur has quit IRC | 01:29 | |
| *** slaweq has quit IRC | 01:30 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove k8s-openstack-provider jobs for repo refresh. https://review.openstack.org/550271 | 01:31 |
| *** jamesmca_ has quit IRC | 01:33 | |
| *** jamesmcarthur has joined #openstack-infra | 01:34 | |
| *** slaweq has joined #openstack-infra | 01:36 | |
| fungi | ianw: clarkb: i vaguely remember it thinks reindexing is unnecessary unless you're upgrading... may need to add a --force flag or something? check our notes from the most recent rename maintenances i guess | 01:36 |
| *** jamesmcarthur has quit IRC | 01:38 | |
| *** slaweq has quit IRC | 01:41 | |
| *** camunoz has quit IRC | 01:42 | |
| *** jamesmcarthur has joined #openstack-infra | 01:44 | |
| *** zhurong has joined #openstack-infra | 01:45 | |
| *** owalsh_ has joined #openstack-infra | 01:46 | |
| *** slaweq has joined #openstack-infra | 01:46 | |
| pabelanger | ianw: mind a +3 https://review.openstack.org/549889/ | 01:47 |
| clarkb | fungi: may not even be necessary if everything else is working | 01:47 |
| *** david-lyle has quit IRC | 01:47 | |
| *** jamesmcarthur has quit IRC | 01:49 | |
| ianw | i think project-config is out of date on nb03 and the jenkins removal etc is confusing it ... re runnign the builds | 01:49 |
| *** owalsh has quit IRC | 01:49 | |
| *** jlabarre has quit IRC | 01:50 | |
| *** gouthamr has quit IRC | 01:50 | |
| *** owalsh_ has quit IRC | 01:51 | |
| *** slaweq has quit IRC | 01:51 | |
| *** agopi has quit IRC | 01:52 | |
| *** owalsh has joined #openstack-infra | 01:52 | |
| openstackgerrit | Merged openstack-infra/bindep master: Update to fedora-27 for testing https://review.openstack.org/549889 | 01:56 |
| *** dave-mccowan has joined #openstack-infra | 01:56 | |
| *** slaweq has joined #openstack-infra | 01:56 | |
| *** owalsh has quit IRC | 01:57 | |
| *** owalsh has joined #openstack-infra | 02:01 | |
| *** slaweq has quit IRC | 02:01 | |
| *** pahuang has quit IRC | 02:02 | |
| *** imacdonn has quit IRC | 02:02 | |
| *** imacdonn has joined #openstack-infra | 02:03 | |
| *** olaph1 has joined #openstack-infra | 02:03 | |
| *** olaph has quit IRC | 02:03 | |
| *** aeng has quit IRC | 02:03 | |
| *** mriedem_afk has quit IRC | 02:03 | |
| *** dingyichen has joined #openstack-infra | 02:04 | |
| *** slaweq has joined #openstack-infra | 02:06 | |
| *** jamesmcarthur has joined #openstack-infra | 02:08 | |
| *** slaweq has quit IRC | 02:11 | |
| *** harlowja has quit IRC | 02:13 | |
| *** pahuang has joined #openstack-infra | 02:14 | |
| *** aeng has joined #openstack-infra | 02:16 | |
| *** slaweq has joined #openstack-infra | 02:17 | |
| *** chrisyang_0660 has joined #openstack-infra | 02:19 | |
| *** slaweq has quit IRC | 02:21 | |
| *** edmondsw has quit IRC | 02:21 | |
| *** edmondsw has joined #openstack-infra | 02:22 | |
| *** germs has joined #openstack-infra | 02:25 | |
| *** rlandy has quit IRC | 02:25 | |
| *** markvoelker has quit IRC | 02:25 | |
| *** markvoelker has joined #openstack-infra | 02:26 | |
| *** edmondsw has quit IRC | 02:26 | |
| *** slaweq has joined #openstack-infra | 02:27 | |
| *** germs has quit IRC | 02:29 | |
| *** olaph has joined #openstack-infra | 02:30 | |
| *** olaph1 has quit IRC | 02:31 | |
| *** slaweq has quit IRC | 02:32 | |
| *** slaweq has joined #openstack-infra | 02:37 | |
| *** smarunich has joined #openstack-infra | 02:41 | |
| *** ramishra has joined #openstack-infra | 02:41 | |
| *** slaweq has quit IRC | 02:42 | |
| *** masuberu has quit IRC | 02:42 | |
| *** andreas_s has joined #openstack-infra | 02:43 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added new endpoint delete venue room https://review.openstack.org/550311 | 02:43 |
| *** annp has joined #openstack-infra | 02:44 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove fetch-zuul-cloner from base job https://review.openstack.org/513506 | 02:44 |
| *** namnh has joined #openstack-infra | 02:47 | |
| *** slaweq has joined #openstack-infra | 02:47 | |
| *** andreas_s has quit IRC | 02:48 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added new endpoint delete venue room https://review.openstack.org/550311 | 02:50 |
| *** gongysh has joined #openstack-infra | 02:50 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Add arm64 support to mirror setup https://review.openstack.org/550312 | 02:51 |
| *** mikal has quit IRC | 02:51 | |
| ianw | ^ this might be one of those things i wrote a whole big thing for then realised it can be done in two lines | 02:51 |
| *** slaweq has quit IRC | 02:52 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/storyboard master: WIP DNM pep8 on ARM64 https://review.openstack.org/549319 | 02:53 |
| *** markvoelker has quit IRC | 02:53 | |
| *** mikal has joined #openstack-infra | 02:53 | |
| *** markvoelker has joined #openstack-infra | 02:53 | |
| *** salv-orl_ has joined #openstack-infra | 02:57 | |
| *** slaweq has joined #openstack-infra | 02:57 | |
| *** salv-orlando has quit IRC | 03:00 | |
| *** masber has joined #openstack-infra | 03:02 | |
| *** slaweq has quit IRC | 03:03 | |
| *** mikal_ has joined #openstack-infra | 03:05 | |
| *** mikal has quit IRC | 03:07 | |
| *** caphrim007_ has quit IRC | 03:07 | |
| *** slaweq has joined #openstack-infra | 03:08 | |
| *** ihrachys_ has quit IRC | 03:09 | |
| *** ihrachys_ has joined #openstack-infra | 03:09 | |
| *** smarunich has quit IRC | 03:12 | |
| *** slaweq has quit IRC | 03:12 | |
| openstackgerrit | Emmet Hikory proposed openstack-infra/project-config master: Build wheels for Ubuntu Xenial ARM64 https://review.openstack.org/549644 | 03:16 |
| *** slaweq has joined #openstack-infra | 03:18 | |
| *** slaweq has quit IRC | 03:22 | |
| *** wolverineav has quit IRC | 03:23 | |
| *** jogo has quit IRC | 03:25 | |
| ianw | infra-root: if anyone around for a review on https://review.openstack.org/#/c/550312/ which modifies the path to ubuntu-ports when setting up mirror; i'm interested if it works. as it's a protected job not sure how to test | 03:27 |
| jhesketh | ianw: lgtm | 03:31 |
| *** masber has quit IRC | 03:36 | |
| persia | Oh, it not working is a zuul feature, rather than it not working. That makes more sense than the job appearing to totally ignore the new file :) | 03:38 |
| ianw | persia: yeah, i know we've had a few ideas floating for testing of protected jobs, but i'm not sure on the current status | 03:42 |
| *** eernst has joined #openstack-infra | 03:43 | |
| *** wolverineav has joined #openstack-infra | 03:43 | |
| *** eernst has quit IRC | 03:45 | |
| dmsimard | ianw: I wonder if we'd be able to tell if we "refused" to run untrusted content and have fallen back to trusted content | 03:45 |
| dmsimard | and put some warning/dancing banana | 03:45 |
| *** eernst has joined #openstack-infra | 03:45 | |
| persia | A warning would be helpful. I've been confused a couple times when Depends-on: didn't do what I expected. | 03:47 |
| dmsimard | Just recently several users have asked about this. It's not straightforward because sometimes it's something that's run by the base playbook, for example.. it's not as simple as saying "it's only about jobs running in project-config, z-j and o-z-j" | 03:47 |
| *** masber has joined #openstack-infra | 03:47 | |
| dmsimard | iirc it was someone testing repository changes (in the zuul-jobs/configure-mirror role) against devstack, but since configure-mirror is part of base, it didn't speculatively test the change | 03:48 |
| dmsimard | just making the links back in my head was sorta challenging and I'm familiar with the "hierarchy" | 03:48 |
| *** slaweq has joined #openstack-infra | 03:48 | |
| *** wolverineav has quit IRC | 03:48 | |
| ianw | yeah, it would be better to just fail out | 03:49 |
| *** jogo has joined #openstack-infra | 03:49 | |
| dmsimard | ianw: Do you want to formalize that in a story ? I was going to bed :p | 03:49 |
| dmsimard | otherwise I can do it tomorrow | 03:49 |
| ianw | i can have a look around and see if there's anything else on it | 03:50 |
| *** slaweq has quit IRC | 03:53 | |
| *** gouthamr has joined #openstack-infra | 03:55 | |
| *** gouthamr has quit IRC | 03:55 | |
| *** ramishra has quit IRC | 03:55 | |
| *** ramishra has joined #openstack-infra | 03:56 | |
| *** links has joined #openstack-infra | 03:59 | |
| *** slaweq has joined #openstack-infra | 03:59 | |
| *** slaweq has quit IRC | 04:03 | |
| persia | ianw: Trying to understand https://review.openstack.org/550312 : while the idea is very appealing to me (as it ought just do the right thing), won't that end up being overwritten by something from https://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/mirror-info/templates/mirror_info.sh.j2 ? | 04:06 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Handle repos with no tags in version-from-git https://review.openstack.org/550175 | 04:06 |
| *** dave-mccowan has quit IRC | 04:08 | |
| *** hongbin has quit IRC | 04:15 | |
| *** nicolasbock has quit IRC | 04:21 | |
| *** ihrachys__ has joined #openstack-infra | 04:23 | |
| *** mikal_ has quit IRC | 04:25 | |
| *** mikal has joined #openstack-infra | 04:25 | |
| *** ihrachys_ has quit IRC | 04:25 | |
| *** Swimingly has quit IRC | 04:25 | |
| *** links has quit IRC | 04:25 | |
| *** germs has joined #openstack-infra | 04:26 | |
| *** germs has quit IRC | 04:26 | |
| *** germs has joined #openstack-infra | 04:26 | |
| *** Swimingly has joined #openstack-infra | 04:26 | |
| *** chrisyang_0660 has quit IRC | 04:27 | |
| *** links has joined #openstack-infra | 04:27 | |
| *** germs has quit IRC | 04:31 | |
| *** eernst has joined #openstack-infra | 04:32 | |
| openstackgerrit | gongysh proposed openstack-infra/irc-meetings master: According to poll, change tacker project meeting date to tuesday https://review.openstack.org/550326 | 04:36 |
| *** masuberu has joined #openstack-infra | 04:36 | |
| *** slaweq has joined #openstack-infra | 04:39 | |
| *** wolverineav has joined #openstack-infra | 04:40 | |
| *** masber has quit IRC | 04:40 | |
| *** psachin has joined #openstack-infra | 04:40 | |
| *** masber has joined #openstack-infra | 04:40 | |
| *** pgadiya has joined #openstack-infra | 04:41 | |
| *** masuberu has quit IRC | 04:42 | |
| *** wolverineav has quit IRC | 04:44 | |
| *** slaweq has quit IRC | 04:44 | |
| *** wolverineav has joined #openstack-infra | 04:46 | |
| *** slaweq has joined #openstack-infra | 04:49 | |
| *** wolverineav has quit IRC | 04:51 | |
| *** slaweq has quit IRC | 04:54 | |
| *** jamesmcarthur has quit IRC | 04:55 | |
| *** ihrachys__ has quit IRC | 04:56 | |
| *** gongysh has quit IRC | 04:57 | |
| *** gongysh has joined #openstack-infra | 04:59 | |
| *** slaweq has joined #openstack-infra | 05:00 | |
| *** slaweq has quit IRC | 05:04 | |
| *** zhurong has quit IRC | 05:08 | |
| *** edmondsw has joined #openstack-infra | 05:09 | |
| *** slaweq has joined #openstack-infra | 05:10 | |
| *** udesale has joined #openstack-infra | 05:10 | |
| *** edmondsw has quit IRC | 05:14 | |
| *** slaweq has quit IRC | 05:15 | |
| *** VW has quit IRC | 05:18 | |
| *** jaosorior has joined #openstack-infra | 05:18 | |
| *** VW has joined #openstack-infra | 05:18 | |
| *** slaweq has joined #openstack-infra | 05:20 | |
| *** VW has quit IRC | 05:22 | |
| *** slaweq has quit IRC | 05:24 | |
| *** eernst has quit IRC | 05:26 | |
| *** dhajare has joined #openstack-infra | 05:27 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add arm64 support to mirror setup https://review.openstack.org/550312 | 05:28 |
| ianw | persia: no, mirror_info.sh is just a informational script that can be sourced on the host if you need mirror paths | 05:29 |
| *** aeng has quit IRC | 05:31 | |
| *** masuberu has joined #openstack-infra | 05:32 | |
| *** masber has quit IRC | 05:35 | |
| *** aeng has joined #openstack-infra | 05:37 | |
| *** claudiub has joined #openstack-infra | 05:46 | |
| *** ykarel has joined #openstack-infra | 05:49 | |
| *** janki has joined #openstack-infra | 05:55 | |
| *** snapiri has quit IRC | 05:56 | |
| *** Goneri has quit IRC | 05:56 | |
| *** david-lyle has joined #openstack-infra | 05:59 | |
| *** bhavik has joined #openstack-infra | 05:59 | |
| *** cshastri has joined #openstack-infra | 06:00 | |
| ianw | persia: cool, mirror setup seems to work fine per comment in 549319. just have to wait for ports to finish syncing now | 06:00 |
| ianw | AJaeger: thanks for pointing out the arm64 image status yesterday, new builds done : http://nl01.openstack.org/dib-image-list | 06:03 |
| *** gongysh has quit IRC | 06:04 | |
| *** gongysh has joined #openstack-infra | 06:05 | |
| openstackgerrit | zhongshengping proposed openstack-infra/project-config master: Remove puppet-openstack-libraries-jobs jobs https://review.openstack.org/550339 | 06:05 |
| openstackgerrit | zhongshengping proposed openstack-infra/openstack-zuul-jobs master: Remove migrated legacy puppet openstack libraries jobs https://review.openstack.org/550340 | 06:09 |
| *** e0ne has joined #openstack-infra | 06:11 | |
| *** abhishekk has joined #openstack-infra | 06:14 | |
| abhishekk | hi any idea why 'http://specs.openstack.org/openstack/glance-specs' is not accessible | 06:14 |
| abhishekk | glance-specs directory is not listed under 'http://specs.openstack.org/openstack' | 06:14 |
| *** e0ne has quit IRC | 06:15 | |
| *** rcernin_ has joined #openstack-infra | 06:18 | |
| *** ykarel has quit IRC | 06:20 | |
| *** rcernin has quit IRC | 06:21 | |
| *** slaweq has joined #openstack-infra | 06:21 | |
| *** slaweq has quit IRC | 06:26 | |
| *** germs has joined #openstack-infra | 06:27 | |
| *** germs has quit IRC | 06:27 | |
| *** germs has joined #openstack-infra | 06:27 | |
| *** kiennt26 has joined #openstack-infra | 06:31 | |
| *** germs has quit IRC | 06:31 | |
| *** slaweq has joined #openstack-infra | 06:31 | |
| *** slaweq has quit IRC | 06:37 | |
| AJaeger | ianw: great! | 06:38 |
| *** alexchadin has joined #openstack-infra | 06:38 | |
| *** masuberu has quit IRC | 06:38 | |
| AJaeger | abhishekk: interesting - question for infra-root. There's an apache error | 06:38 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Publish everything but keystone to PyPI https://review.openstack.org/531825 | 06:39 |
| openstackgerrit | zhongshengping proposed openstack-infra/project-config master: Remove puppet-openstack-libraries-jobs jobs https://review.openstack.org/550339 | 06:39 |
| abhishekk | AJaeger, could you please paste the logs for reference? | 06:39 |
| AJaeger | mordred: if you want to continue this, I resolved merge conflicts and check for missing pypi - we miss heat, glance, manila, swift, tripleo-quickstart, tripleo-quickstart-extras, zaqar-ui in pypi | 06:39 |
| AJaeger | abhishekk: I'm not a root | 06:39 |
| AJaeger | mordred: do you want to create those? | 06:40 |
| abhishekk | AJaeger, ohh, thank you :D | 06:40 |
| *** ihrachys has joined #openstack-infra | 06:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Publish everything but keystone,congress,magnum to PyPI https://review.openstack.org/531825 | 06:40 |
| openstackgerrit | zhongshengping proposed openstack-infra/openstack-zuul-jobs master: Remove migrated legacy puppet openstack libraries jobs https://review.openstack.org/550340 | 06:40 |
| *** links has quit IRC | 06:41 | |
| *** slaweq has joined #openstack-infra | 06:42 | |
| *** ihrachys_ has joined #openstack-infra | 06:42 | |
| *** bhavik has quit IRC | 06:43 | |
| AJaeger | abhishekk, infra-root, found problem - using .htaccess ;( | 06:44 |
| *** jamesdenton has quit IRC | 06:44 | |
| AJaeger | patch coming... | 06:44 |
| abhishekk | AJaeger, thank you for quick response | 06:45 |
| *** ihrachys has quit IRC | 06:45 | |
| *** slaweq has quit IRC | 06:46 | |
| *** olaph1 has joined #openstack-infra | 06:48 | |
| *** olaph has quit IRC | 06:48 | |
| *** zhurong has joined #openstack-infra | 06:49 | |
| AJaeger | infra-root, version-from-git is broken, see http://paste.openstack.org/show/693544/ - do we need to revert https://review.openstack.org/550175 ? | 06:49 |
| AJaeger | log file for project-config change at http://logs.openstack.org/25/531825/4/check/openstack-zuul-jobs-linters/4d58782/job-output.txt.gz#_2018-03-07_06_44_35_262594 | 06:49 |
| *** olaph has joined #openstack-infra | 06:50 | |
| *** links has joined #openstack-infra | 06:51 | |
| *** slaweq has joined #openstack-infra | 06:52 | |
| *** olaph1 has quit IRC | 06:52 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix version-from-git syntax https://review.openstack.org/550357 | 06:52 |
| AJaeger | fix is here ^ | 06:53 |
| AJaeger | tobiash, frickler, in case you're around to unblock project-config changes, please review ^ | 06:53 |
| ianw | bah, sorry | 06:55 |
| tobiash | AJaeger: looking | 06:55 |
| *** prometheanfire has quit IRC | 06:56 | |
| *** slaweq has quit IRC | 06:57 | |
| tobiash | +3 | 06:57 |
| *** edmondsw has joined #openstack-infra | 06:58 | |
| *** coolsvap has joined #openstack-infra | 07:00 | |
| AJaeger | thanks | 07:00 |
| *** edmondsw has quit IRC | 07:03 | |
| *** rcernin has joined #openstack-infra | 07:05 | |
| *** ykarel has joined #openstack-infra | 07:05 | |
| *** rcernin_ has quit IRC | 07:06 | |
| *** threestrands_ has joined #openstack-infra | 07:09 | |
| *** threestrands_ has quit IRC | 07:09 | |
| *** threestrands_ has joined #openstack-infra | 07:09 | |
| *** threestrands has quit IRC | 07:10 | |
| *** ykarel has quit IRC | 07:11 | |
| abhishekk | AJaeger, any idea how much time it takes for publishing after merging the specs patch? | 07:13 |
| *** threestrands_ has quit IRC | 07:13 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix version-from-git syntax https://review.openstack.org/550357 | 07:20 |
| *** andreas_s has joined #openstack-infra | 07:21 | |
| *** cloud-liang has joined #openstack-infra | 07:31 | |
| *** slaweq has joined #openstack-infra | 07:33 | |
| *** cloud-liang has quit IRC | 07:35 | |
| *** ykarel has joined #openstack-infra | 07:37 | |
| *** slaweq has quit IRC | 07:38 | |
| *** kjackal has joined #openstack-infra | 07:42 | |
| *** slaweq has joined #openstack-infra | 07:43 | |
| *** cloud-liang has joined #openstack-infra | 07:47 | |
| *** slaweq has quit IRC | 07:48 | |
| *** armaan has joined #openstack-infra | 07:51 | |
| *** hashar has joined #openstack-infra | 07:53 | |
| *** slaweq has joined #openstack-infra | 07:53 | |
| *** pcaruana has joined #openstack-infra | 07:54 | |
| *** pcaruana has quit IRC | 07:55 | |
| *** pcaruana has joined #openstack-infra | 07:55 | |
| AJaeger | abhishekk: check zuul.openstack.org | 07:55 |
| AJaeger | abhishekk: once the post job is finished, it takes 5 mins to sync | 07:56 |
| *** e0ne has joined #openstack-infra | 07:56 | |
| *** slaweq has quit IRC | 07:58 | |
| *** gus has quit IRC | 08:02 | |
| *** adriant has quit IRC | 08:03 | |
| *** adriant has joined #openstack-infra | 08:03 | |
| *** slaweq has joined #openstack-infra | 08:04 | |
| *** gus has joined #openstack-infra | 08:04 | |
| *** olivierbourdon3- has quit IRC | 08:05 | |
| *** alexchadin has quit IRC | 08:07 | |
| *** jpena|off is now known as jpena | 08:07 | |
| *** slaweq has quit IRC | 08:08 | |
| *** olivierbourdon38 has joined #openstack-infra | 08:09 | |
| *** alexchadin has joined #openstack-infra | 08:10 | |
| *** slaweq has joined #openstack-infra | 08:14 | |
| frickler | AJaeger: isn't the logic in https://review.openstack.org/#/c/550357/1/roles/version-from-git/tasks/main.yaml still broken? | 08:15 |
| frickler | AJaeger: L57 checks "when: commits_since_tag_output.rc == 0" but L64 has "when: not commits_since_tag_output.rc != 0" which seems to be the same conditional to me, not the opposite as I would expect | 08:15 |
| *** slaweq has quit IRC | 08:18 | |
| *** aviau has quit IRC | 08:19 | |
| *** aviau has joined #openstack-infra | 08:19 | |
| *** e0ne has quit IRC | 08:19 | |
| AJaeger | frickler: indeed, that'S double negation and looks wrong ;( | 08:20 |
| AJaeger | what a rat hole ;( | 08:20 |
| *** danpawlik has joined #openstack-infra | 08:21 | |
| *** jtomasek has joined #openstack-infra | 08:22 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix condition in version-from-git https://review.openstack.org/550388 | 08:22 |
| AJaeger | tobiash, ianw, frickler ^ | 08:22 |
| *** tesseract has joined #openstack-infra | 08:22 | |
| *** jtomasek has quit IRC | 08:22 | |
| *** dingyichen has quit IRC | 08:23 | |
| *** slaweq has joined #openstack-infra | 08:24 | |
| *** slaweq_ has joined #openstack-infra | 08:24 | |
| tobiash | AJaeger: +2 with comment | 08:24 |
| *** jtomasek has joined #openstack-infra | 08:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix condition in version-from-git https://review.openstack.org/550388 | 08:25 |
| frickler | pabelanger: yay, tempest-py36 ran successfully now on bionic http://logs.openstack.org/04/548504/3/check/tempest-full-py36/2bc193f/job-output.txt.gz | 08:25 |
| AJaeger | tobiash: fixed nit - thanks | 08:25 |
| tobiash | +2 | 08:26 |
| AJaeger | frickler, pabelanger \o/ | 08:26 |
| *** germs has joined #openstack-infra | 08:27 | |
| *** germs has quit IRC | 08:27 | |
| *** germs has joined #openstack-infra | 08:27 | |
| *** slaweq_ has quit IRC | 08:29 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: DNM: run tox-cover job https://review.openstack.org/550392 | 08:32 |
| *** threestrands has joined #openstack-infra | 08:32 | |
| *** threestrands has quit IRC | 08:32 | |
| *** threestrands has joined #openstack-infra | 08:32 | |
| *** germs has quit IRC | 08:32 | |
| *** shardy has joined #openstack-infra | 08:34 | |
| *** slaweq_ has joined #openstack-infra | 08:34 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Support pagination in the installation map https://review.openstack.org/550393 | 08:35 |
| *** threestrands has quit IRC | 08:36 | |
| aspiers | infra-root: any idea why https://governance.openstack.org/sigs/ isn't showing the Upgrade SIG even though it's in sigs.yaml? | 08:37 |
| *** jaosorior_ has joined #openstack-infra | 08:37 | |
| AJaeger | aspiers: you best check the post file from a recent merge | 08:38 |
| AJaeger | Which repo is publishing that? | 08:38 |
| *** slaweq_ has quit IRC | 08:39 | |
| *** jaosorior has quit IRC | 08:40 | |
| *** florianf has joined #openstack-infra | 08:41 | |
| AJaeger | aspiers: I think a couple of commits merged at the same time and all published at the same time - and UPgrade SIG was in the last change, thus got overwritten ;( wait for the next change to merge... | 08:42 |
| *** cshastri has quit IRC | 08:42 | |
| *** rcernin has quit IRC | 08:42 | |
| *** cshastri has joined #openstack-infra | 08:43 | |
| *** slaweq_ has joined #openstack-infra | 08:44 | |
| *** edmondsw has joined #openstack-infra | 08:46 | |
| *** electrofelix has joined #openstack-infra | 08:47 | |
| *** ralonsoh has joined #openstack-infra | 08:48 | |
| *** ralonsoh_ has joined #openstack-infra | 08:48 | |
| *** slaweq_ has quit IRC | 08:49 | |
| *** edmondsw has quit IRC | 08:50 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix condition in version-from-git https://review.openstack.org/550388 | 08:51 |
| *** slaweq_ has joined #openstack-infra | 08:55 | |
| aspiers | AJaeger: https://review.openstack.org/#/c/548203/ | 08:55 |
| aspiers | AJaeger: conveniently enough, I posted another change more recently :) https://review.openstack.org/#/c/550094/ | 08:56 |
| aspiers | but I think only ttx can +2 that | 08:56 |
| abhishekk | AJaeger, its working now, thank you | 08:57 |
| aspiers | AJaeger: how can commits get merged at the same time? surely it's a serial process? | 08:57 |
| AJaeger | aspiers: merged seconds after each other. The post process is not serial | 08:58 |
| *** jamesmcarthur has joined #openstack-infra | 08:58 | |
| ttx | aspiers: let me approve that second one | 08:58 |
| *** slaweq_ has quit IRC | 08:59 | |
| ttx | aspiers: that should sync alright | 08:59 |
| *** lucas-afk is now known as lucasagomes | 08:59 | |
| *** kjackal has quit IRC | 09:00 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: DNM: run tox-cover job https://review.openstack.org/550392 | 09:01 |
| *** jamesmcarthur has quit IRC | 09:02 | |
| *** jaosorior_ is now known as jaosorior | 09:03 | |
| *** slaweq_ has joined #openstack-infra | 09:05 | |
| *** witek has joined #openstack-infra | 09:06 | |
| aspiers | AJaeger: got it | 09:06 |
| aspiers | ttx: thanks, I'm watching zuul ... | 09:07 |
| aspiers | ttx: do you know if there is any plan to convert LCOO to a SIG? | 09:08 |
| aspiers | publish-governance-sigs is running now: http://zuul.openstack.org/stream.html?uuid=0ce3b1c3be064f30998fc4edbc0653a4&logfile=console.log | 09:09 |
| *** slaweq_ has quit IRC | 09:09 | |
| aspiers | these jobs seem to spend much more time installing stuff like bindeps than actually running the job... I wonder if they could be drastically sped up with judicious use of containers | 09:10 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Remove Storlets team meeting https://review.openstack.org/550064 | 09:11 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Remove Product WG meeting https://review.openstack.org/550164 | 09:11 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: According to poll, change tacker project meeting date to tuesday https://review.openstack.org/550326 | 09:11 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Remove Rally team meeting https://review.openstack.org/550061 | 09:11 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Tenant config can be read from an external script https://review.openstack.org/535878 | 09:12 |
| aspiers | AJaeger, ttx: OK, Upgrade SIG is now visible on https://governance.openstack.org/sigs/ | 09:12 |
| aspiers | thanks both! | 09:12 |
| witek | hi, we have a post openstack-cover-job in our project | 09:14 |
| witek | https://github.com/openstack-infra/project-config/blob/master/zuul.d/projects.yaml#L8510 | 09:14 |
| witek | where can I see output of its execution? | 09:14 |
| AJaeger | aspiers: add a custom bindep.txt file that just installs what you need - see what we use here http://git.openstack.org/cgit/openstack/security-doc/tree/bindep.txt - you might remove gettext as well if you have no translations | 09:14 |
| *** slaweq_ has joined #openstack-infra | 09:15 | |
| AJaeger | witek: That's one of the reasons why I suggest to not do this anymore ;) | 09:15 |
| AJaeger | witek: https://docs.openstack.org/infra/manual/developers.html#post-processing | 09:15 |
| AJaeger | or: zuul.openstack.org/builds.html?job_name=openstack-tox-cover&project=openstack/monasca-api | 09:16 |
| *** felipemonteiro has joined #openstack-infra | 09:18 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Revert "Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header"" https://review.openstack.org/514489 | 09:19 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add Ansible version to job header https://review.openstack.org/532304 | 09:19 |
| *** slaweq_ has quit IRC | 09:19 | |
| *** olivierbourdon38 has quit IRC | 09:20 | |
| *** calbers has joined #openstack-infra | 09:21 | |
| witek | AJaeger: thanks, I didn't know the `build` tab, nice | 09:23 |
| *** armaan has quit IRC | 09:24 | |
| *** owalsh has quit IRC | 09:24 | |
| *** owalsh has joined #openstack-infra | 09:25 | |
| *** armaan has joined #openstack-infra | 09:25 | |
| *** slaweq_ has joined #openstack-infra | 09:25 | |
| *** ihrachys_ has quit IRC | 09:25 | |
| *** ihrachys_ has joined #openstack-infra | 09:26 | |
| *** slaweq_ has quit IRC | 09:29 | |
| ttx | aspiers: re:LCOO we talked to them but it's a bit unclear what their focus is | 09:30 |
| ttx | It's a bit of a closed group | 09:30 |
| ttx | I'd rather focus on Public Cloud WG, Large deployments (=Scaling SIG?), and Telco/NFV | 09:31 |
| *** gongysh has quit IRC | 09:32 | |
| ttx | which is easier to relate to than a "large contributing organization" group that does not really include the top largest contributing organizations | 09:33 |
| *** olivierbourdon38 has joined #openstack-infra | 09:34 | |
| *** gongysh has joined #openstack-infra | 09:34 | |
| *** olivierbourdon38 has quit IRC | 09:35 | |
| *** slaweq_ has joined #openstack-infra | 09:35 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add horizon to required-projects https://review.openstack.org/550130 | 09:35 |
| *** ihrachys_ has quit IRC | 09:36 | |
| *** armaan has quit IRC | 09:38 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add horizon to required-projects https://review.openstack.org/550130 | 09:38 |
| *** armaan has joined #openstack-infra | 09:38 | |
| *** slaweq_ has quit IRC | 09:40 | |
| *** derekh has joined #openstack-infra | 09:42 | |
| *** verdurin has joined #openstack-infra | 09:45 | |
| *** dtantsur|afk is now known as dtantsur | 09:45 | |
| *** slaweq_ has joined #openstack-infra | 09:45 | |
| AJaeger | witek: do you want to keep the cover job as post job or remove it? Does it give you any benefit? Your call... | 09:45 |
| witek | I thought about moving it to gate pipeline | 09:46 |
| *** olivierbourdon38 has joined #openstack-infra | 09:47 | |
| *** efoley has joined #openstack-infra | 09:48 | |
| openstackgerrit | Dirk Mueller proposed openstack-infra/project-config master: Drop kingbird, networking-hpe and trio2o from g-r sync https://review.openstack.org/550415 | 09:48 |
| *** namnh has quit IRC | 09:49 | |
| *** slaweq_ has quit IRC | 09:50 | |
| AJaeger | witek: if it's voting, sure | 09:50 |
| *** slaweq_ has joined #openstack-infra | 09:56 | |
| *** kiennt26 has quit IRC | 09:57 | |
| *** slaweq_ has quit IRC | 10:00 | |
| *** tosky has joined #openstack-infra | 10:03 | |
| *** stakeda has quit IRC | 10:04 | |
| *** e0ne has joined #openstack-infra | 10:04 | |
| *** Douhet has quit IRC | 10:09 | |
| *** rkukura_ has joined #openstack-infra | 10:10 | |
| *** Douhet has joined #openstack-infra | 10:10 | |
| *** warlord77[m] has joined #openstack-infra | 10:10 | |
| *** rkukura has quit IRC | 10:11 | |
| *** rkukura_ is now known as rkukura | 10:11 | |
| *** eyalb has joined #openstack-infra | 10:12 | |
| *** auristor has quit IRC | 10:15 | |
| *** auristor has joined #openstack-infra | 10:16 | |
| *** slaweq_ has joined #openstack-infra | 10:16 | |
| *** ralonsoh_ has quit IRC | 10:17 | |
| *** kukacz has quit IRC | 10:17 | |
| frickler | AJaeger: infra-root: any idea why chef jobs are failing badly in emit-job-header today? will look further myself, too http://logs.openstack.org/46/549346/4/gate/openstack-chef-repo-rake/e69bc4d/job-output.txt.gz#_2018-03-07_09_42_32_429881 | 10:20 |
| *** slaweq_ has quit IRC | 10:21 | |
| eyalb | I have retry limit in all my jobs and it has the same reason of nodepool undefined | 10:21 |
| warlord77[m] | hi I would like to contribute to openstack infra | 10:22 |
| warlord77[m] | can anyone tell me where to start | 10:22 |
| dulek | frickler: I'm hitting the same thing. | 10:23 |
| *** zhurong has quit IRC | 10:23 | |
| frickler | dulek: do you have a sample job? | 10:23 |
| *** annp has quit IRC | 10:23 | |
| dulek | frickler: Nope, it's happening in all kuryr-kubernetes jobs, so I guess it's a global thing. | 10:24 |
| AJaeger | frickler: https://review.openstack.org/532304 and https://review.openstack.org/514489 merged - is that related? | 10:24 |
| frickler | dulek: infra-root: looking at backlog, https://review.openstack.org/514489 and https://review.openstack.org/532304 look suspicous | 10:25 |
| *** pbourke has quit IRC | 10:25 | |
| frickler | AJaeger: yep, probably | 10:25 |
| AJaeger | warlord77[m]: glad to hear - best to discuss with the US based team once they are awake | 10:25 |
| *** pbourke has joined #openstack-infra | 10:25 | |
| AJaeger | frickler: let's remove nodepool for now? | 10:26 |
| *** slaweq_ has joined #openstack-infra | 10:26 | |
| AJaeger | {{ nodepool.provider |default('N/A') }} - why is that not working? | 10:26 |
| frickler | AJaeger: I don't see any obvious solution, guess I'll go for a revert of those two | 10:27 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 10:27 |
| AJaeger | frickler: patch coming | 10:28 |
| frickler | AJaeger: k | 10:28 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix emit-job-header https://review.openstack.org/550425 | 10:28 |
| *** germs has joined #openstack-infra | 10:28 | |
| *** germs has quit IRC | 10:28 | |
| *** germs has joined #openstack-infra | 10:28 | |
| AJaeger | frickler, tobiash ^ | 10:28 |
| andreaf | ianw: hopefully addressing your comments https://review.openstack.org/#/c/548642/ | 10:28 |
| andreaf | AJaeger frickler zuul-jobs review needed for multinode ^^^ | 10:29 |
| AJaeger | infra-root, please check https://review.openstack.org/550425 - why is nodepool undefined? | 10:29 |
| AJaeger | andreaf: thanks, will review later tonight if nobody beats me to it ;) | 10:30 |
| warlord77[m] | AJaeger: can u tell me where to start ? | 10:30 |
| frickler | AJaeger: I think it is intentional and the default doesn't work as expected, your patch indeed seems to be the correct solution | 10:30 |
| *** slaweq_ has quit IRC | 10:31 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add horizon to required-projects https://review.openstack.org/550130 | 10:31 |
| *** yamamoto has quit IRC | 10:31 | |
| AJaeger | warlord77[m]: https://docs.openstack.org/infra/ is our docs root - but I really suggest to wait and discuss based on your preferences and knowledge. Some info on help needed is at https://governance.openstack.org/tc/reference/help-most-needed.html#community-infrastructure-sysadmins - and fungi should be around in 4 to 5 hours, best talk with him. He has a better overview on where help is needed. | 10:32 |
| andreaf | AJaeger thanks | 10:33 |
| *** germs has quit IRC | 10:33 | |
| *** felipemonteiro has quit IRC | 10:33 | |
| *** edmondsw has joined #openstack-infra | 10:34 | |
| AJaeger | dulek, frickler, once 550425 is merged, please recheck - and tell whether it works/fails here. | 10:35 |
| dulek | AJaeger: Will do, thanks! | 10:35 |
| *** fifilip has joined #openstack-infra | 10:36 | |
| *** slaweq_ has joined #openstack-infra | 10:36 | |
| AJaeger | andreaf: you might need to recheck your change once 550425 is merged | 10:37 |
| *** edmondsw has quit IRC | 10:38 | |
| andreaf | AJaeger cool thanks | 10:38 |
| andreaf | AJaeger did you get my messages yesterday about the irrelevant-files discussion at the PTG? | 10:39 |
| *** olaph1 has joined #openstack-infra | 10:40 | |
| *** kazsh has quit IRC | 10:40 | |
| *** slaweq_ has quit IRC | 10:41 | |
| *** olaph has quit IRC | 10:41 | |
| AJaeger | andreaf: yes, thanks - saw in backscroll. That's what I feared as outcome ;) | 10:42 |
| *** kazsh has joined #openstack-infra | 10:45 | |
| *** slaweq_ has joined #openstack-infra | 10:47 | |
| *** coolsvap has quit IRC | 10:49 | |
| *** slaweq_ has quit IRC | 10:51 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix emit-job-header https://review.openstack.org/550425 | 10:53 |
| AJaeger | tobiash, frickler I still see nodepool not defined in https://review.openstack.org/#/c/550425/ - let'S remove it completely ^ | 10:53 |
| AJaeger | frickler, tobiash, I'm also fine with reverting the two changes... - and it's lunch time now, so please take over... | 10:55 |
| AJaeger | (but I fear it's lunch time everywhere in Germany ;) | 10:55 |
| tobiash | I'm at lunch too | 10:55 |
| tobiash | ;) | 10:55 |
| AJaeger | tobiash: enjoy! | 10:56 |
| * AJaeger does not understand why nodepool is defined for some jobs but not for all | 10:56 | |
| tobiash | U2 | 10:57 |
| openstackgerrit | Emmet Hikory proposed openstack-infra/openstack-zuul-jobs master: Vary Ubuntu mirror path based on architecture https://review.openstack.org/546434 | 10:59 |
| *** tedlit has joined #openstack-infra | 11:00 | |
| andreaf | corvus I left a comment on https://review.openstack.org/#/c/549252/ - ianw frickler this may use more devstack reviews | 11:01 |
| *** gnuoy has joined #openstack-infra | 11:02 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 11:03 |
| *** lathiat has quit IRC | 11:03 | |
| andreaf | AJaeger I rebased https://review.openstack.org/#/c/548642/ on top of your patch - ianw tobiash frickler | 11:04 |
| rm_work | erg, so a ton of gates are broken because of whatever it is you guys are working on fixing right now, right? so I shouldn't bother with rechecks? | 11:06 |
| rm_work | the nodepool thing | 11:06 |
| rm_work | http://paste.openstack.org/show/693592/ | 11:07 |
| *** neiljerram has joined #openstack-infra | 11:08 | |
| frickler | rm_work: yeah, fix is in the gates, pls bear with us for a moment | 11:08 |
| rm_work | yeah just double-checking | 11:09 |
| * rm_work bears | 11:09 | |
| neiljerram | Morning all! What does it mean if check jobs are getting RETRY_LIMIT? (Apologies if that is already being discussed here; my log only goes back two lines.) | 11:10 |
| * beisner strolls through, waves, wishes luck | 11:10 | |
| frickler | neiljerram: known issue, should be fixed in about half an hour hopefully | 11:10 |
| *** gongysh has quit IRC | 11:10 | |
| beisner | thx all | 11:10 |
| neiljerram | frickler, ack, thanks. | 11:10 |
| *** abhishekk has quit IRC | 11:12 | |
| openstackgerrit | Emmet Hikory proposed openstack-infra/project-config master: Build wheels for Ubuntu Xenial ARM64 https://review.openstack.org/549644 | 11:12 |
| *** janki has quit IRC | 11:15 | |
| frickler | AJaeger: infra-root: looks like the patch is still failing, because it isn't self-testing? http://logs.openstack.org/25/550425/2/check/build-sphinx-docs/d874f65/job-output.txt.gz /me thinks to force-merge it | 11:16 |
| *** cgoncalves has joined #openstack-infra | 11:17 | |
| *** gongysh has joined #openstack-infra | 11:17 | |
| mordred | frickler: ++ | 11:17 |
| *** udesale_ has joined #openstack-infra | 11:19 | |
| *** shardy has quit IRC | 11:20 | |
| *** shardy has joined #openstack-infra | 11:21 | |
| *** udesale has quit IRC | 11:21 | |
| *** shardy has quit IRC | 11:21 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix emit-job-header https://review.openstack.org/550425 | 11:21 |
| *** gongysh has quit IRC | 11:21 | |
| *** udesale has joined #openstack-infra | 11:21 | |
| *** shardy has joined #openstack-infra | 11:22 | |
| frickler | #status log force-merged https://review.openstack.org/550425 to unblock CI | 11:22 |
| openstackstatus | frickler: finished logging | 11:22 |
| *** fifilip has quit IRC | 11:23 | |
| *** udesale_ has quit IRC | 11:24 | |
| frickler | infra-root: rechecked job passed the critical step, seems we are good again | 11:25 |
| mordred | frickler: woot | 11:27 |
| frickler | rm_work: dulek: neiljerram: you should be able to recheck now | 11:27 |
| rm_work | ok, thanks! | 11:27 |
| *** slaweq_ has joined #openstack-infra | 11:27 | |
| *** salv-orl_ has quit IRC | 11:29 | |
| dulek | frickler: Doing that now, thanks! | 11:29 |
| *** salv-orlando has joined #openstack-infra | 11:29 | |
| *** nicolasbock has joined #openstack-infra | 11:31 | |
| *** yamamoto has joined #openstack-infra | 11:32 | |
| *** slaweq_ has quit IRC | 11:32 | |
| *** salv-orlando has quit IRC | 11:34 | |
| frickler | AJaeger: infra-root: so now someone should ponder how we can improve our testing here. or was this also a thing that should have been gone via the base-test playbook? | 11:34 |
| frickler | s/gone/going/ | 11:34 |
| mordred | frickler: yes, that is a thing that should have gone through base-test | 11:35 |
| *** eyalb has left #openstack-infra | 11:35 | |
| mordred | frickler: although I thought we actually had the functional tests to cover that role | 11:35 |
| mordred | I think the underlying issue is that it only breaks in a specific scenario (no nodes) - which we still might not have caught with either base-test or the functional tests | 11:36 |
| mordred | frickler: maybe we want to add something to the zuul-jobs functional tests that runs things in a no-node scenario? | 11:36 |
| *** udesale has quit IRC | 11:36 | |
| *** slaweq_ has joined #openstack-infra | 11:38 | |
| *** yamamoto has quit IRC | 11:38 | |
| tobiash | mordred: can we by default also test base-test on zuul-jobs changes? | 11:40 |
| tobiash | I don't feel that good anymore when approving stuff in zuul-jobs :/ | 11:40 |
| *** ldnunes has joined #openstack-infra | 11:42 | |
| *** slaweq_ has quit IRC | 11:42 | |
| *** alexchadin has quit IRC | 11:43 | |
| *** yamamoto has joined #openstack-infra | 11:45 | |
| *** jamesmcarthur has joined #openstack-infra | 11:46 | |
| *** salv-orlando has joined #openstack-infra | 11:48 | |
| *** slaweq_ has joined #openstack-infra | 11:48 | |
| neiljerram | frickler, thanks | 11:50 |
| andreaf | AJaeger looking at https://review.openstack.org/#/c/548642/, it may be that tox-py35-on-zuul is not doing the right thing when processing depends-on? | 11:50 |
| *** jamesmcarthur has quit IRC | 11:50 | |
| andreaf | AJaeger granted, being the same repo I could just have rebased it on top, however it should still work | 11:50 |
| *** yamamoto has quit IRC | 11:50 | |
| AJaeger | mordred: it failed sphinx-docs job - is that a no-node scenario? | 11:51 |
| *** slaweq_ has quit IRC | 11:52 | |
| AJaeger | andreaf: I think it's the problem we had recently - and this should have been done via base-test. Basically, we do not use the change in this context. | 11:53 |
| *** katkapilatova has joined #openstack-infra | 11:54 | |
| *** katkapilatova has left #openstack-infra | 11:54 | |
| *** lathiat has joined #openstack-infra | 11:54 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add horizon to required-projects https://review.openstack.org/550130 | 11:54 |
| *** lucasagomes is now known as lucas-hungry | 11:54 | |
| andreaf | AJaeger oh, so is that fixed by another patch? if we are not testing the current patch we'll keep breaking the gate | 11:54 |
| *** slaweq_ has joined #openstack-infra | 11:58 | |
| AJaeger | andreaf: No, we cannot test such things this way. corvus explained that a few days ago in detail, sorry, don't find a reference right now ;( | 11:58 |
| fungi | see the comment starting at http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/jobs.yaml#n7 | 11:59 |
| *** olaph has joined #openstack-infra | 11:59 | |
| AJaeger | andreaf: found it - http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-03-02.log.html#t2018-03-02T15:05:48 | 11:59 |
| AJaeger | and as fungi says ;) | 11:59 |
| *** alexchadin has joined #openstack-infra | 12:00 | |
| *** olaph1 has quit IRC | 12:00 | |
| *** panda|off is now known as panda | 12:00 | |
| andreaf | AJaeger fungi ok thanks | 12:01 |
| *** beisner has quit IRC | 12:02 | |
| *** csatari has quit IRC | 12:02 | |
| *** slaweq_ has quit IRC | 12:03 | |
| *** jpena is now known as jpena|lunch | 12:03 | |
| *** beisner has joined #openstack-infra | 12:03 | |
| *** csatari has joined #openstack-infra | 12:03 | |
| *** slaweq_ has joined #openstack-infra | 12:08 | |
| *** fifilip has joined #openstack-infra | 12:09 | |
| tosky | uhm, are we really safe with rechecking? | 12:12 |
| tosky | I think that I rechecked after it was given green light again here, but: https://review.openstack.org/#/c/550300/ | 12:12 |
| *** janki has joined #openstack-infra | 12:13 | |
| *** slaweq_ has quit IRC | 12:13 | |
| AJaeger | tosky: The log files started earlier than the merge for that change | 12:16 |
| AJaeger | tosky: so, let's recheck, plesae | 12:16 |
| tosky | ack, thanks | 12:17 |
| *** slaweq_ has joined #openstack-infra | 12:17 | |
| *** ykarel_ has joined #openstack-infra | 12:18 | |
| *** salv-orl_ has joined #openstack-infra | 12:18 | |
| *** claudiub|2 has joined #openstack-infra | 12:18 | |
| *** ykarel has quit IRC | 12:18 | |
| *** ykarel_ has quit IRC | 12:18 | |
| *** ykarel_ has joined #openstack-infra | 12:18 | |
| *** slaweq__ has joined #openstack-infra | 12:18 | |
| *** myoung|ruck|off is now known as myoung|ruck | 12:19 | |
| *** bkero- has quit IRC | 12:20 | |
| *** bkero- has joined #openstack-infra | 12:20 | |
| *** panda is now known as panda|lunch | 12:20 | |
| *** claudiub has quit IRC | 12:20 | |
| *** yolanda has quit IRC | 12:20 | |
| *** salv-orlando has quit IRC | 12:20 | |
| *** slaweq has quit IRC | 12:20 | |
| *** yolanda has joined #openstack-infra | 12:21 | |
| *** mattoliverau has quit IRC | 12:21 | |
| *** mattoliverau has joined #openstack-infra | 12:22 | |
| *** slaweq__ has quit IRC | 12:23 | |
| *** jlabarre has joined #openstack-infra | 12:23 | |
| *** hjensas has joined #openstack-infra | 12:24 | |
| *** gfidente has joined #openstack-infra | 12:25 | |
| *** gfidente has joined #openstack-infra | 12:25 | |
| *** jamesdenton has joined #openstack-infra | 12:25 | |
| *** armaan has quit IRC | 12:26 | |
| *** armaan has joined #openstack-infra | 12:26 | |
| *** psachin has quit IRC | 12:27 | |
| *** slaweq has joined #openstack-infra | 12:29 | |
| *** germs has joined #openstack-infra | 12:29 | |
| *** germs has quit IRC | 12:29 | |
| *** germs has joined #openstack-infra | 12:29 | |
| *** alexchadin has quit IRC | 12:29 | |
| AJaeger | mordred: btw. I added some comments on https://etherpad.openstack.org/p/zuulv3-remove-zuul-cloner - I think it's time to continue this effort. | 12:31 |
| *** nicolasbock has quit IRC | 12:31 | |
| AJaeger | mordred: see also https://review.openstack.org/#/c/531825/ | 12:31 |
| *** dave-mccowan has joined #openstack-infra | 12:32 | |
| *** slaweq has quit IRC | 12:33 | |
| *** germs has quit IRC | 12:34 | |
| *** alexchadin has joined #openstack-infra | 12:34 | |
| *** slaweq has joined #openstack-infra | 12:39 | |
| *** alexchadin has quit IRC | 12:39 | |
| *** udesale has joined #openstack-infra | 12:39 | |
| *** lbragstad has joined #openstack-infra | 12:39 | |
| *** alexchadin has joined #openstack-infra | 12:42 | |
| *** slaweq has quit IRC | 12:44 | |
| *** alexchadin has quit IRC | 12:47 | |
| *** felipemonteiro has joined #openstack-infra | 12:47 | |
| *** cloud-liang has quit IRC | 12:49 | |
| *** slaweq has joined #openstack-infra | 12:49 | |
| *** claudiub|2 has quit IRC | 12:52 | |
| *** cshastri has quit IRC | 12:53 | |
| *** slaweq has quit IRC | 12:54 | |
| *** wolverineav has joined #openstack-infra | 12:54 | |
| *** olaph1 has joined #openstack-infra | 12:55 | |
| *** armaan has quit IRC | 12:55 | |
| *** armaan has joined #openstack-infra | 12:56 | |
| *** olaph has quit IRC | 12:56 | |
| *** Nil_ has quit IRC | 12:56 | |
| *** alexchadin has joined #openstack-infra | 12:58 | |
| *** panda|lunch is now known as panda | 12:58 | |
| *** olaph1 is now known as olaph | 12:59 | |
| *** myoung|ruck is now known as myoung|ruck|mtg | 12:59 | |
| *** slaweq has joined #openstack-infra | 12:59 | |
| *** wolverineav has quit IRC | 12:59 | |
| *** claudiub has joined #openstack-infra | 13:00 | |
| *** cshastri has joined #openstack-infra | 13:01 | |
| *** lucas-hungry is now known as lucasagomes | 13:01 | |
| *** claudiub|2 has joined #openstack-infra | 13:01 | |
| *** yamamoto has joined #openstack-infra | 13:02 | |
| *** alexchadin has quit IRC | 13:03 | |
| *** alexchadin has joined #openstack-infra | 13:03 | |
| *** slaweq has quit IRC | 13:03 | |
| *** jpena|lunch is now known as jpena | 13:05 | |
| *** fifilip has quit IRC | 13:05 | |
| *** claudiub has quit IRC | 13:05 | |
| *** fifilip has joined #openstack-infra | 13:06 | |
| *** yamamoto has quit IRC | 13:07 | |
| *** alexchadin has quit IRC | 13:07 | |
| *** slaweq has joined #openstack-infra | 13:09 | |
| *** jlabarre has quit IRC | 13:13 | |
| *** slaweq has quit IRC | 13:14 | |
| *** alexchadin has joined #openstack-infra | 13:15 | |
| *** rlandy has joined #openstack-infra | 13:19 | |
| *** slaweq has joined #openstack-infra | 13:19 | |
| *** prometheanfire has joined #openstack-infra | 13:20 | |
| *** jlabarre has joined #openstack-infra | 13:21 | |
| *** slaweq has quit IRC | 13:24 | |
| *** cgoncalves has left #openstack-infra | 13:25 | |
| *** edmondsw_ has joined #openstack-infra | 13:25 | |
| *** hjensas has quit IRC | 13:25 | |
| *** jlabarre has quit IRC | 13:27 | |
| *** slaweq has joined #openstack-infra | 13:30 | |
| mordred | AJaeger: yah - I think it's also time to continue the "rework publication in base job" effort | 13:30 |
| *** dprince has joined #openstack-infra | 13:31 | |
| AJaeger | mordred: I won't have time to drive any of this but happy to help a bit, see my two reviews. | 13:31 |
| mordred | AJaeger: I think we're done with lines 1-13 in that etherpad | 13:33 |
| *** slaweq has quit IRC | 13:34 | |
| *** pgadiya has quit IRC | 13:35 | |
| AJaeger | mordred: agreed | 13:37 |
| mordred | AJaeger: left some comments on the castellan-ui patch - looks good in general | 13:38 |
| AJaeger | mordred: looking at the two cahnges, I wondered where to add the requirements (test-requirements, requirements) and what this means for developer running locally (they would get the pip version instead of git checkout) | 13:38 |
| *** slaweq has joined #openstack-infra | 13:40 | |
| mordred | AJaeger: yes - I thnk they will get the pip version locally - but that just becomes another instance of the general problem such as "I'm working on a change to openstacksdk that depends on an unlanded change in keystoneauth, how to I test it locally" | 13:40 |
| mordred | I started work on tooling to allow people to be more easily able to re-create tox-siblings locally | 13:41 |
| *** VW has joined #openstack-infra | 13:41 | |
| *** VW has joined #openstack-infra | 13:44 | |
| AJaeger | ah, good plan! | 13:44 |
| AJaeger | mordred: want to send a patch to add neutron and horizon to global requirements? | 13:45 |
| *** slaweq has quit IRC | 13:45 | |
| openstackgerrit | eldad marciano proposed openstack-infra/grafyaml master: Add gauge caps for singlestat panel. https://review.openstack.org/550474 | 13:45 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers https://review.openstack.org/520192 | 13:45 |
| AJaeger | mordred: updated castellan-ui change | 13:45 |
| mordred | AJaeger: yah... one sec | 13:47 |
| *** tedlit_ has joined #openstack-infra | 13:47 | |
| rm_work | hey, question -- is there something wrong with our playbooks for our checks? it looks like our tests failed but zuul is showing SUCCESS: http://logs.openstack.org/82/550282/6/check/neutron-lbaasv2-dsvm-py3x-scenario-namespace/0b166b4/job-output.txt.gz#_2018-03-07_13_33_04_973753 | 13:47 |
| *** esberglu has joined #openstack-infra | 13:48 | |
| rm_work | this is for neutron-lbaas CR 550282,6 which is in check now | 13:48 |
| rm_work | oh nm | 13:49 |
| rm_work | i misread the job name | 13:49 |
| rm_work | that one did fail | 13:49 |
| *** cgoncalves has joined #openstack-infra | 13:49 | |
| * rm_work derps | 13:49 | |
| *** tedlit has quit IRC | 13:50 | |
| *** slaweq has joined #openstack-infra | 13:50 | |
| *** trown|outtypewww is now known as trown | 13:54 | |
| *** slaweq has quit IRC | 13:54 | |
| mordred | AJaeger: remote: https://review.openstack.org/550475 Add neutron and horizon to global requirements | 13:55 |
| clarkb | PTG recap sent out. I'm sure I missed stuff so feel free to add to it | 13:56 |
| clarkb | mordred: also in writing that I realized that I think jim being sick plus you sneaking out early resulted in us never having a real zuul support for containers discussion | 13:56 |
| clarkb | would probably be a good one to take to the zuul list | 13:57 |
| *** olaph1 has joined #openstack-infra | 13:59 | |
| *** olaph has quit IRC | 14:00 | |
| *** slaweq has joined #openstack-infra | 14:00 | |
| *** yamamoto has joined #openstack-infra | 14:03 | |
| *** myoung|ruck|mtg is now known as myoung|ruck | 14:03 | |
| *** jlabarre has joined #openstack-infra | 14:04 | |
| *** slaweq has quit IRC | 14:04 | |
| *** efoley has quit IRC | 14:05 | |
| *** jlabarre has quit IRC | 14:05 | |
| *** jlabarre has joined #openstack-infra | 14:06 | |
| mordred | clarkb: agree | 14:06 |
| *** efoley has joined #openstack-infra | 14:07 | |
| *** germs has joined #openstack-infra | 14:08 | |
| *** germs has quit IRC | 14:08 | |
| *** germs has joined #openstack-infra | 14:08 | |
| *** germs has quit IRC | 14:09 | |
| *** yamamoto has quit IRC | 14:09 | |
| *** germs has joined #openstack-infra | 14:09 | |
| *** germs has quit IRC | 14:09 | |
| *** germs has joined #openstack-infra | 14:09 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Support pagination in the installation map https://review.openstack.org/550393 | 14:09 |
| *** slaweq has joined #openstack-infra | 14:10 | |
| *** felipemonteiro has quit IRC | 14:11 | |
| *** esberglu has quit IRC | 14:14 | |
| *** claudiub has joined #openstack-infra | 14:14 | |
| *** slaweq has quit IRC | 14:15 | |
| *** lbragstad has quit IRC | 14:15 | |
| *** armaan has quit IRC | 14:16 | |
| *** claudiub|2 has quit IRC | 14:17 | |
| *** armaan has joined #openstack-infra | 14:20 | |
| *** slaweq has joined #openstack-infra | 14:21 | |
| *** lbragstad has joined #openstack-infra | 14:21 | |
| *** trown is now known as trown|rover | 14:21 | |
| *** olaph has joined #openstack-infra | 14:22 | |
| *** armaan has quit IRC | 14:22 | |
| clarkb | Gerrit queue is still looking good | 14:23 |
| *** olaph1 has quit IRC | 14:23 | |
| *** slaweq has quit IRC | 14:25 | |
| clarkb | re that nodepool dict not existing and causing problems in jobs. Maybe we should put an empty nodepool dict in the inventory for every job? | 14:27 |
| clarkb | though I guess none of the keys owuld exist so we'd still get an error if not checking contents | 14:28 |
| *** esberglu has joined #openstack-infra | 14:29 | |
| AJaeger | mordred: thanks | 14:31 |
| *** slaweq has joined #openstack-infra | 14:31 | |
| mordred | clarkb: yah - BUT - I think the issue in this case is that nodepool.provider | default('n/a') failed on nodepool not being defined, so it didn't even get to provider not being defined and thus taking on the default value | 14:31 |
| mordred | clarkb: so the code in question tried to be defensive, but wasn't defensive enough | 14:31 |
| *** erlon has joined #openstack-infra | 14:32 | |
| mordred | I think a set_fact: nodepool: {} when: nodepool is not defined at the top would have made the nodepool.provider | default('n/a') work | 14:33 |
| erlon | @all, does ZNC bouncer with freenode? | 14:33 |
| *** ihrachys has joined #openstack-infra | 14:33 | |
| clarkb | mordred: seems like an ansible bug where default should work along that tree | 14:33 |
| *** germs has quit IRC | 14:33 | |
| erlon | *does anyone uses... | 14:33 |
| mnaser | erlon: i used to, but i started using irccloud and i like it | 14:33 |
| clarkb | mordred: but that does make me think my original idea of just having an empty dict in each inventory would be a good idea | 14:33 |
| *** efoley has quit IRC | 14:33 | |
| clarkb | mordred: so that its less conscious thought required for users and default() will just work for them | 14:34 |
| erlon | mnaser, Im going the oposite way, irc cloud is eating all my memory | 14:34 |
| clarkb | erlon: I think dansmith may use it | 14:34 |
| mordred | clarkb: yah | 14:34 |
| erlon | mnaser, like, all!! | 14:34 |
| mnaser | erlon: os x? i run this desktop client which seems to be doing okay | 14:34 |
| erlon | mnaser, linux | 14:34 |
| mnaser | https://github.com/irccloud/irccloud-desktop ? | 14:34 |
| *** kjackal has joined #openstack-infra | 14:35 | |
| dansmith | yeah I use znc | 14:35 |
| erlon | let me see if they have a version for linux | 14:35 |
| *** slaweq has quit IRC | 14:35 | |
| mordred | mnaser: there's a desktop client for irccloud? wow. ... you'd think just exposing an out-bound irc port users could connect to would be the choice... | 14:35 |
| mnaser | mordred: yeah the world works in funny ways :) | 14:36 |
| *** Nil_ has joined #openstack-infra | 14:36 | |
| *** links has quit IRC | 14:37 | |
| openstackgerrit | Merged openstack-infra/zuul master: Support pagination in the installation map https://review.openstack.org/550393 | 14:37 |
| erlon | dansmith, dansmith can you share your conf? mine still not accepting the SASL authentication though I have set and load the module | 14:38 |
| *** r-daneel has joined #openstack-infra | 14:38 | |
| mnaser | https://wiki.znc.in/Sasl is what i followed a while back | 14:38 |
| dansmith | erlon: my conf has credentials in it, so, no, and it's also for a very old version | 14:38 |
| dansmith | I'm just using TLS and plaintext auth | 14:39 |
| *** agopi has joined #openstack-infra | 14:39 | |
| clarkb | mordred: looking at it closer (rtfsing) nodepool is a host var attribute and I guess we don't set host vars on localhost | 14:39 |
| clarkb | mordred: but we always set a nodepool dict otherwise | 14:39 |
| clarkb | mordred: can we safely set hostvars on localhost? | 14:40 |
| mordred | clarkb: not really, no - we'd have to put localhost into the inventory to do that, and if we did that hosts: all would start matching localhost | 14:41 |
| mordred | (localhost is special as it is now) | 14:41 |
| *** slaweq has joined #openstack-infra | 14:41 | |
| clarkb | ok I was worried about that | 14:42 |
| clarkb | if instead we set a top level nodepool dict to {} will the hostvars be more specific and override as necessary? | 14:43 |
| erlon | dansmith, hmmm, you run those commands from your local client every time you connect? | 14:43 |
| dansmith | erlon: what commands? but no, I don't run anything when I connect | 14:43 |
| erlon | dansmith, the link you sent, is a series of /something commands | 14:44 |
| dansmith | erlon: that was mnaser | 14:45 |
| erlon | opss, mnaser ^ | 14:45 |
| pabelanger | frickler: yah, wheels built too, I'll be publishing them today (bionic) | 14:45 |
| *** mriedem has joined #openstack-infra | 14:45 | |
| *** slaweq has quit IRC | 14:45 | |
| mnaser | erlon: nope, you do it once and it gets stored | 14:47 |
| *** agopi has quit IRC | 14:47 | |
| clarkb | ansible doesn't seem to have the idea of a global variable which maybe isn't a bad thing | 14:49 |
| clarkb | but makes the just set nodepool to {} more difficult | 14:49 |
| *** electrofelix has quit IRC | 14:51 | |
| *** cshastri has quit IRC | 14:51 | |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Create new project for oslo.limit https://review.openstack.org/550491 | 14:51 |
| *** slaweq has joined #openstack-infra | 14:51 | |
| *** olaph1 has joined #openstack-infra | 14:52 | |
| clarkb | mordred: maybe the best thing is a set fact if nodepool is not set in the base job | 14:53 |
| *** ramishra has quit IRC | 14:54 | |
| mordred | clarkb: well - that won't persist into other playbooks | 14:54 |
| *** olaph has quit IRC | 14:54 | |
| *** erlon_ has joined #openstack-infra | 14:54 | |
| clarkb | oh right each playbook is a new process | 14:54 |
| mordred | clarkb: we might just have to work around it in that playbook directly | 14:55 |
| pabelanger | Yah, default() filters won't work with dicts. Found that out recently | 14:55 |
| mordred | clarkb: I'm not sure there is a great answer for this particular thing otherwise (what a fascinating edge-case :) ) | 14:55 |
| pabelanger | so, need a more complex jinja2 expression | 14:55 |
| *** slaweq has quit IRC | 14:56 | |
| *** claudiub|2 has joined #openstack-infra | 14:56 | |
| *** erlon has quit IRC | 14:57 | |
| *** erlon_ is now known as erlon | 14:57 | |
| *** efoley has joined #openstack-infra | 14:57 | |
| *** claudiub has quit IRC | 14:59 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add nodepool provider back to job header https://review.openstack.org/550493 | 15:01 |
| mordred | pabelanger, clarkb: ^^ something like that | 15:01 |
| *** slaweq has joined #openstack-infra | 15:01 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Bring publish-wheel-mirror-ubuntu-bionic job online https://review.openstack.org/550494 | 15:02 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Create new project for oslo.limit https://review.openstack.org/550491 | 15:02 |
| *** pabelanger has quit IRC | 15:04 | |
| *** migi has quit IRC | 15:04 | |
| *** yamamoto has joined #openstack-infra | 15:05 | |
| AJaeger | mordred: can we test this with our base-test jobs, please? | 15:05 |
| mordred | AJaeger: totally | 15:05 |
| *** jamesmcarthur has joined #openstack-infra | 15:06 | |
| *** slaweq has quit IRC | 15:06 | |
| *** claudiub has joined #openstack-infra | 15:06 | |
| *** bobh has joined #openstack-infra | 15:06 | |
| *** bnemec has joined #openstack-infra | 15:08 | |
| *** claudiub|2 has quit IRC | 15:09 | |
| *** rlandy has quit IRC | 15:09 | |
| *** yamamoto has quit IRC | 15:09 | |
| *** rlandy has joined #openstack-infra | 15:09 | |
| *** rlandy has quit IRC | 15:10 | |
| *** claudiub|2 has joined #openstack-infra | 15:11 | |
| *** slaweq has joined #openstack-infra | 15:12 | |
| *** eharney has quit IRC | 15:12 | |
| *** edmondsw_ is now known as edmondsw | 15:13 | |
| AJaeger | mordred, prometheanfire, seems some more discussion about background of https://review.openstack.org/550475 is needed | 15:13 |
| *** claudiub has quit IRC | 15:14 | |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Create new project for oslo.limit https://review.openstack.org/550491 | 15:15 |
| prometheanfire | glean? | 15:15 |
| prometheanfire | oh, that one | 15:16 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/openstack-zuul-jobs master: Fix ceph py35 job definition https://review.openstack.org/550497 | 15:16 |
| mriedem | mtreinish: andreaf: ^ is a start | 15:16 |
| *** slaweq has quit IRC | 15:16 | |
| prometheanfire | ya, I thought it was generally considered a nono to use a top level project as a lib | 15:16 |
| *** pabelanger has joined #openstack-infra | 15:17 | |
| clarkb | prometheanfire: I think largely because we weren't publishing them for other reasons | 15:17 |
| *** eernst has joined #openstack-infra | 15:18 | |
| clarkb | prometheanfire: but everyone did it anyways and we got over our concerns with publishing them | 15:18 |
| prometheanfire | I thought it was to avoid complicated and deep dep trees | 15:18 |
| pabelanger | okay, http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64/ now exists in AFS | 15:18 |
| prometheanfire | pabelanger: :D | 15:18 |
| pabelanger | Hmm, emit-header is failing lint | 15:18 |
| pabelanger | http://logs.openstack.org/94/550494/1/check/openstack-zuul-jobs-linters/176bea6/job-output.txt.gz#_2018-03-07_15_09_28_660155 | 15:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Fix lint issue with emit-job-header role https://review.openstack.org/550501 | 15:20 |
| pabelanger | tobiash: AJaeger: dmsimard: mordred: ^fixes project-config | 15:21 |
| tobiash | hm, today is break the jobs day :( | 15:21 |
| dmsimard | the dangers of merging base job things.. | 15:22 |
| AJaeger | Yeah! | 15:22 |
| *** jamesmcarthur has quit IRC | 15:22 | |
| *** slaweq has joined #openstack-infra | 15:22 | |
| dmsimard | I'm still unhappy that we really want to speculatively test things within a "trusted" context but we can't | 15:22 |
| tobiash | pabelanger: should we do the test-base dance for this? | 15:22 |
| dmsimard | leading to awkward "hacks" like base-test | 15:22 |
| tobiash | or is it urgent? | 15:22 |
| dmsimard | tobiash: we can't | 15:22 |
| dmsimard | tobiash: because emit-job-header is in base already | 15:22 |
| dmsimard | tobiash: the base-test dance only works when you're *adding* a new role to the base job or modifying the base playbook | 15:23 |
| tobiash | ok, so 550501 really looks innocent | 15:23 |
| *** alexchadin has quit IRC | 15:24 | |
| tobiash | +2 from me | 15:24 |
| *** pcichy has joined #openstack-infra | 15:24 | |
| pabelanger | yah | 15:24 |
| tobiash | but I think I should avoid hitting +A today on zuul-jobs... | 15:24 |
| tobiash | bad karma | 15:24 |
| tobiash | so I leave it to someone other to break the base job this time ;) | 15:25 |
| *** wolverineav has joined #openstack-infra | 15:25 | |
| *** slaweq has quit IRC | 15:26 | |
| pabelanger | tobiash: np, I can +3 | 15:28 |
| *** owalsh_ has joined #openstack-infra | 15:28 | |
| *** wolverineav has quit IRC | 15:29 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/storyboard master: Make notification driver configurable https://review.openstack.org/538574 | 15:30 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/storyboard master: Add MQTT notification publisher https://review.openstack.org/538575 | 15:30 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/storyboard master: Add configurable notification subscriber and mqtt driver https://review.openstack.org/540958 | 15:30 |
| dmsimard | tobiash: eh, it happens.. not the end of the world and it's actually a good reminder that we need to address the lack of testing coverage for that kind of things (and I'll refer you back to my previous rambling just now) | 15:30 |
| *** germs has joined #openstack-infra | 15:31 | |
| *** germs has quit IRC | 15:31 | |
| *** germs has joined #openstack-infra | 15:31 | |
| *** owalsh has quit IRC | 15:31 | |
| *** armaan has joined #openstack-infra | 15:32 | |
| *** slaweq has joined #openstack-infra | 15:32 | |
| *** slaweq_ has quit IRC | 15:32 | |
| *** germs has quit IRC | 15:32 | |
| *** slaweq_ has joined #openstack-infra | 15:33 | |
| openstackgerrit | Javier Peña proposed openstack-infra/system-config master: Move AFS mirror code to puppet-openstackci https://review.openstack.org/529032 | 15:34 |
| cgoncalves | hi. is there some way one could get notified of failing periodic jobs? | 15:34 |
| clarkb | cgoncalves: yes, you can subscribe to them on openstack health using rss | 15:35 |
| *** ramishra has joined #openstack-infra | 15:35 | |
| *** germs has joined #openstack-infra | 15:35 | |
| *** germs has quit IRC | 15:35 | |
| *** germs has joined #openstack-infra | 15:35 | |
| clarkb | http://status.openstack.org/openstack-health/#/ | 15:36 |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support https://review.openstack.org/529376 | 15:36 |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support for RHEL/CentOS https://review.openstack.org/528739 | 15:36 |
| *** slaweq has quit IRC | 15:37 | |
| *** ramishra has quit IRC | 15:37 | |
| *** slaweq_ has quit IRC | 15:38 | |
| cgoncalves | clarkb: I see a rss per project. no rss per job per project? | 15:38 |
| *** trown|rover is now known as trown|brb | 15:38 | |
| clarkb | cgoncalves: if you change the group in the top right to build_name it changes the way its presented and allows you to subscribe per job | 15:40 |
| *** hongbin has joined #openstack-infra | 15:40 | |
| cgoncalves | clarkb: awesome, thanks! | 15:40 |
| *** slaweq has joined #openstack-infra | 15:42 | |
| *** agopi has joined #openstack-infra | 15:45 | |
| *** udesale has quit IRC | 15:45 | |
| *** trown|brb is now known as trown|rover | 15:46 | |
| *** slaweq has quit IRC | 15:46 | |
| *** salv-orlando has joined #openstack-infra | 15:48 | |
| *** jamesmcarthur has joined #openstack-infra | 15:48 | |
| *** salv-orl_ has quit IRC | 15:50 | |
| *** slaweq has joined #openstack-infra | 15:52 | |
| pabelanger | AJaeger: mordred: thanks for pushing on tox-siblings | 15:52 |
| *** e0ne has quit IRC | 15:53 | |
| *** e0ne has joined #openstack-infra | 15:54 | |
| AJaeger | pabelanger: just a tiny step - see the etherpad, there's lot of work to do IMHO | 15:54 |
| *** david-lyle has quit IRC | 15:54 | |
| pabelanger | AJaeger: :( for a lot of work. But happy to help | 15:55 |
| *** myoung|ruck is now known as myoung|ruck|brb | 15:55 | |
| *** slaweq has quit IRC | 15:57 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix lint issue with emit-job-header role https://review.openstack.org/550501 | 15:58 |
| *** gouthamr has joined #openstack-infra | 16:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Make an entry for zuul-web-test https://review.openstack.org/550516 | 16:02 |
| *** slaweq has joined #openstack-infra | 16:02 | |
| mordred | pabelanger, clarkb, dmsimard: ^^ that's my next step in figuring out the zuul-web sitch | 16:03 |
| *** caphrim007 has joined #openstack-infra | 16:03 | |
| clarkb | mordred: tobiash responded to comments on https://review.openstack.org/#/c/550493/1 and rechecked it | 16:04 |
| mordred | tobiash: oh - yif your syntax works I like it | 16:06 |
| *** yamamoto has joined #openstack-infra | 16:06 | |
| tobiash | mordred: I tested this locally | 16:06 |
| *** eharney has joined #openstack-infra | 16:06 | |
| *** hemna_ has joined #openstack-infra | 16:06 | |
| mordred | tobiash: neat! | 16:06 |
| *** slaweq has quit IRC | 16:07 | |
| *** slaweq has joined #openstack-infra | 16:08 | |
| *** pcaruana has quit IRC | 16:10 | |
| pabelanger | Zuul busy this morning | 16:11 |
| AJaeger | mordred, https://review.openstack.org/550475 is finally completely ;( | 16:12 |
| *** yamamoto has quit IRC | 16:12 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Add unit test for multiple launchers https://review.openstack.org/540916 | 16:12 |
| *** slaweq_ has joined #openstack-infra | 16:13 | |
| *** agopi has quit IRC | 16:13 | |
| *** slaweq has quit IRC | 16:13 | |
| *** agopi has joined #openstack-infra | 16:13 | |
| *** andreas_s has quit IRC | 16:14 | |
| *** myoung|ruck|brb is now known as myoung|ruck | 16:17 | |
| frickler | dmsimard: tobiash: well, one could test existing roles by cloning them first and using the modified clone in base-test. that may be clumsy, but might still be better than breaking the world | 16:18 |
| *** slaweq_ has quit IRC | 16:18 | |
| *** andreas_s has joined #openstack-infra | 16:19 | |
| dmsimard | frickler: yeah... I've been meaning to try the new nodepool static driver inside a Zuul multinode job (two nodes) where the first node would be a zuul+nodepool installation (with the speculative changes merged in $somehow) and then the second node would be the nodepool static node. We would run $realjob on the nodepool static node and return success/failure based on the result. | 16:19 |
| dmsimard | There's some challenges with my idea (do we load all the upstream config in the nested zuul? we don't have 32GB of RAM, we don't have the private keys for the secrets, etc.) | 16:20 |
| dmsimard | mordred: ^ we may have discussed this at some point, I forget. | 16:21 |
| pabelanger | dmsimard: tobiash does this today | 16:22 |
| clarkb | dmsimard: fwiw you don't need 32GB of ram to load the entirety of our config | 16:22 |
| pabelanger | aside from private keys issue, it does work | 16:22 |
| clarkb | dmsimard: you only need about 2GB based on my last restart when I was paying attention to it | 16:22 |
| dmsimard | clarkb: yeah I was exaggerating | 16:22 |
| mordred | dmsimard: I don't know that we got that far - but I don't think we need to load the entire config if we're only going to run a few specific jobs | 16:22 |
| *** slaweq has joined #openstack-infra | 16:23 | |
| *** rlandy has joined #openstack-infra | 16:23 | |
| pabelanger | but, we don't want to test with production keys, we likely need a flag to not have zuul decrypt secrets on load | 16:23 |
| pabelanger | I didn't know zuul tried doing that on first boot, only after tobiash showed me his jobs | 16:23 |
| dmsimard | mordred: the nested zuul needs to run exactly what the upstream zuul would run and that implies loading all the config and then force-merging (or whatever) the speculative changes in (that would have otherwise not been tested because it was running in a trusted context) | 16:24 |
| clarkb | or just make it fail gracefully | 16:24 |
| clarkb | and then any jobs trying to use the secrets could fail too | 16:24 |
| dmsimard | pabelanger: right, we don't want to test with production keys/secrets. | 16:24 |
| dmsimard | pabelanger: but we want to be able to exercise playbooks and roles that consumes those | 16:24 |
| pabelanger | ideally ya | 16:25 |
| pabelanger | tobiash usecase was to confirm github projects were setup properly | 16:25 |
| dmsimard | So we could somehow provide mock data or whatever, I dunno. | 16:25 |
| *** slaweq has quit IRC | 16:27 | |
| *** andreas_s has quit IRC | 16:28 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add copy of emit-job-header to test header change https://review.openstack.org/550518 | 16:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use test-emit-job-header in base-test https://review.openstack.org/550519 | 16:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add nodepool provider back to job header https://review.openstack.org/550493 | 16:30 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Revert "Use test-emit-job-header in base-test" https://review.openstack.org/550520 | 16:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Remove test-emit-job-header https://review.openstack.org/550521 | 16:31 |
| mordred | tobiash, dmsimard, AJaeger, frickler: ^^ ok -there's a base-test dance with the emit-job-header change | 16:32 |
| * dmsimard dances | 16:32 | |
| mordred | still need a job reparented on the base-test job to test that it works - does anyone remember what a job that failed was? pretty much anything, right? | 16:33 |
| AJaeger | sphinx-docs-job on zuul-jobs for example | 16:33 |
| *** slaweq has joined #openstack-infra | 16:33 | |
| clarkb | anything that did localhost | 16:33 |
| *** andreas_s has joined #openstack-infra | 16:33 | |
| mordred | oh - wait. this is copmletely silly | 16:34 |
| mordred | emit-job-header ALWAYS runs on localhost | 16:34 |
| mordred | - hosts: localhost | 16:34 |
| mordred | roles: | 16:34 |
| mordred | - role: emit-job-header | 16:34 |
| clarkb | well then | 16:35 |
| mordred | so there are no circumstances when, as currently structured, this will be anything other than N/A | 16:35 |
| AJaeger | config-core, here's a simple change to add pep8 to the contributor-guide, could you review, please? https://review.openstack.org/#/c/549281/ | 16:35 |
| * mordred goes to abandon changes | 16:35 | |
| AJaeger | mordred: see https://review.openstack.org/#/c/550425/ - it had some success and some failures. I still don't understand why not all failed... | 16:36 |
| mordred | AJaeger: openstack-infra-base-integration-* don't use the base job - they're special | 16:37 |
| AJaeger | mordred: http://logs.openstack.org/25/550425/1/check/openstack-infra-base-integration-centos-7/bfed456/job-output.txt.gz#_2018-03-07_10_31_03_172042 | 16:37 |
| AJaeger | Provider: ovh-bhs1 | 16:37 |
| AJaeger | mordred: ah, but there it worked ^ | 16:37 |
| AJaeger | mordred: you're right that's it... | 16:38 |
| *** slaweq has quit IRC | 16:38 | |
| *** ykarel_ has quit IRC | 16:38 | |
| pabelanger | config-core: https://review.openstack.org/550494/ reviews to add ubuntu bionic wheels, I've confirmed requirements works as expected and AFS volume is created | 16:38 |
| *** weshay is now known as weshay|PTO | 16:38 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Simplify launcher cleanup worker https://review.openstack.org/545867 | 16:38 |
| *** dhill__ has joined #openstack-infra | 16:38 | |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support https://review.openstack.org/529376 | 16:39 |
| *** dhill_ has quit IRC | 16:39 | |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support for RHEL/CentOS https://review.openstack.org/528739 | 16:39 |
| AJaeger | config-core, could you review andreaf's https://review.openstack.org/#/c/548642/ to help with multi-node devstack? | 16:39 |
| *** olivierbourdon38 has quit IRC | 16:40 | |
| *** danpawlik has quit IRC | 16:41 | |
| *** VW_ has joined #openstack-infra | 16:43 | |
| *** slaweq has joined #openstack-infra | 16:43 | |
| *** tedlit_ has quit IRC | 16:45 | |
| *** VW__ has joined #openstack-infra | 16:46 | |
| *** VW has quit IRC | 16:46 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Add additional builder debug logging https://review.openstack.org/546303 | 16:47 |
| *** VW_ has quit IRC | 16:47 | |
| *** andreas_s has quit IRC | 16:47 | |
| *** slaweq has quit IRC | 16:47 | |
| *** andreas_s has joined #openstack-infra | 16:49 | |
| tobiash | mordred: the emit-job-header should probably run on all with once: true | 16:52 |
| *** olaph has joined #openstack-infra | 16:52 | |
| pabelanger | mordred: clarkb: dmsimard: mind a +3 https://review.openstack.org/550494/ So we can try doing wheel builds for bionic today | 16:52 |
| openstackgerrit | Merged openstack-infra/project-config master: Add pep8 for contributor-guide https://review.openstack.org/549281 | 16:53 |
| *** claudiub|2 is now known as claudiub | 16:53 | |
| tobiash | pabelanger, dmsimard: the challenge with the multinode job is that if you have many repos and big repos there is a scalability problem | 16:53 |
| *** olaph1 has quit IRC | 16:53 | |
| tobiash | except if you manage to use the production mergers for that | 16:53 |
| *** slaweq has joined #openstack-infra | 16:53 | |
| tobiash | (which I don't have yet) | 16:54 |
| tobiash | at least if not all repos fit onto the nde | 16:54 |
| tobiash | node | 16:54 |
| * fungi steps out to grab some timely lunch, bbiab | 16:55 | |
| *** slaweq has quit IRC | 16:57 | |
| dmsimard | tobiash: that's a good point | 16:57 |
| *** slaweq has joined #openstack-infra | 16:57 | |
| *** andreas_s has quit IRC | 16:58 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added endpoints to get venue rooms and venue floors by id https://review.openstack.org/550528 | 16:59 |
| lbragstad | AJaeger: o/ what was the second change you referenced in your review messages? https://review.openstack.org/#/c/550491/3 | 16:59 |
| *** agopi has quit IRC | 17:00 | |
| AJaeger | lbragstad: adding jobs to zuul.d/projects.yaml | 17:00 |
| AJaeger | lbragstad: https://docs.openstack.org/infra/manual/creators.html | 17:01 |
| mnaser | hey uh | 17:01 |
| mnaser | where is configure_mirror.sh go? | 17:01 |
| lbragstad | aha - https://docs.openstack.org/infra/manual/creators.html#add-system-required-template | 17:01 |
| AJaeger | mnaser: dead | 17:01 |
| mnaser | AJaeger: so what's the alternative? | 17:01 |
| AJaeger | mnaser: https://review.openstack.org/548240 | 17:02 |
| clarkb | mnaser: we have a role that does the work of writing out the script to source in jobs and configures things | 17:02 |
| mnaser | ok maybe i miswrote my question | 17:02 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added endpoints to get venue rooms and venue floors by id https://review.openstack.org/550528 | 17:02 |
| mnaser | there was /etc/ci/mirror_info or something | 17:02 |
| mnaser | that we could export and get access to all the mirrors in env | 17:02 |
| clarkb | mnaser: yes that should remain | 17:03 |
| clarkb | iirc the new role writes out that file | 17:03 |
| AJaeger | http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/mirror-info/templates/mirror_info.sh.j2 | 17:03 |
| clarkb | /etc/ci/mirror_info.sh | 17:03 |
| AJaeger | bbl | 17:03 |
| mnaser | AJaeger: clarkb ok perfect thank you | 17:03 |
| *** agopi has joined #openstack-infra | 17:04 | |
| *** agopi has quit IRC | 17:05 | |
| *** danpawlik has joined #openstack-infra | 17:07 | |
| *** yamamoto has joined #openstack-infra | 17:08 | |
| *** danpawlik has quit IRC | 17:12 | |
| openstackgerrit | Ruby Loo proposed openstack-infra/project-config master: Remove ironic-inspector-tempest-plugin https://review.openstack.org/550537 | 17:12 |
| *** yamamoto has quit IRC | 17:13 | |
| *** jamesmcarthur has quit IRC | 17:14 | |
| *** armaan has quit IRC | 17:15 | |
| *** hemna_ has quit IRC | 17:15 | |
| *** armaan has joined #openstack-infra | 17:15 | |
| *** david-lyle has joined #openstack-infra | 17:16 | |
| *** janki has quit IRC | 17:18 | |
| *** dhajare has quit IRC | 17:18 | |
| clarkb | tobiash: the concern is being able to merge and push all of the git repos to all of the nodes in a multinode job? | 17:19 |
| tobiash | clarkb: say if you have 2500 repos with a total of 1tb this won't fit on a test node typically | 17:20 |
| clarkb | tobiash: right but you'd only be pushing the required project right? | 17:20 |
| clarkb | or are you saying all 1tb is required projects? | 17:20 |
| tobiash | clarkb: in my use case I'm gating the tenant config | 17:20 |
| tobiash | so I need all repos | 17:21 |
| tobiash | in case you want to gate zuul-jobs with that a smaller number of repos would be sufficient | 17:21 |
| *** florianf has quit IRC | 17:21 | |
| clarkb | gotcha | 17:21 |
| tobiash | in a multinode job having zuul installed there also the mergers are typically there | 17:22 |
| pabelanger | https://review.openstack.org/550494/ could still use a +3, if people are looking for an easy review | 17:22 |
| tobiash | which is ok if all fits on a test node | 17:22 |
| *** andreas_s has joined #openstack-infra | 17:22 | |
| *** olivierbourdon38 has joined #openstack-infra | 17:23 | |
| *** olivierbourdon38 has quit IRC | 17:23 | |
| clarkb | pabelanger: whats the relationship between the releases project and bionic wheels?? | 17:23 |
| *** olivierbourdon38 has joined #openstack-infra | 17:24 | |
| clarkb | no more slack irc gateway so my suggestion of just pointing irc bots at that gateway for slack users is no longer viable :/ | 17:24 |
| *** olivierbourdon38 has quit IRC | 17:24 | |
| bkero- | You would need to do it the other way around | 17:25 |
| pabelanger | clarkb: they now how jobs intree to try and build wheels when bindep.txt changes, https://review.openstack.org/550253/ and https://review.openstack.org/550254/ is how I tested wheels worked for bionic | 17:25 |
| bkero- | have slack bot be on irc | 17:25 |
| pabelanger | but we only release wheels with the periodic jobs today | 17:25 |
| *** olivierbourdon38 has joined #openstack-infra | 17:26 | |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Create new project for oslo.limit https://review.openstack.org/550491 | 17:26 |
| clarkb | bkero-: right but none of our bots are slack bots they are irc bots and people were asking how to use them with slack (so I suggestd using the gateway) | 17:27 |
| bkero- | yeah | 17:27 |
| bkero- | That would have to change | 17:27 |
| *** cshastri has joined #openstack-infra | 17:27 | |
| *** zoli is now known as zoli|gone | 17:27 | |
| *** zoli|gone is now known as zoli | 17:27 | |
| clarkb | pabelanger: but we should be building wheels independent of what the release team is doing right? looking at those changes maybe you meant requirements? | 17:27 |
| pabelanger | oh, ha | 17:28 |
| pabelanger | yes, requirements team | 17:29 |
| pabelanger | I can update commit message | 17:29 |
| clarkb | pabelanger: its ok I approved it | 17:29 |
| clarkb | just wanted ot make sure we hadn't created some new dependeny we didn't need/want | 17:29 |
| clarkb | requirements makes sense because that is where we get the list of things to make wheels for | 17:29 |
| pabelanger | thanks | 17:29 |
| pabelanger | yah, let me get more coffee | 17:30 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers https://review.openstack.org/520192 | 17:31 |
| *** kjackal has quit IRC | 17:31 | |
| *** yamahata has quit IRC | 17:32 | |
| *** olivierbourdon3- has joined #openstack-infra | 17:32 | |
| *** olivierbourdon3- has quit IRC | 17:32 | |
| *** olivierbourdon3- has joined #openstack-infra | 17:32 | |
| *** david-lyle has quit IRC | 17:33 | |
| *** olivierbourdon38 has quit IRC | 17:33 | |
| openstackgerrit | Paul Bourke (pbourke) proposed openstack-infra/bindep master: Add OracleLinux support https://review.openstack.org/536355 | 17:34 |
| *** fifilip has quit IRC | 17:36 | |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support https://review.openstack.org/529376 | 17:37 |
| openstackgerrit | Javier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support for RHEL/CentOS https://review.openstack.org/528739 | 17:37 |
| *** agopi has joined #openstack-infra | 17:38 | |
| openstackgerrit | Merged openstack-infra/project-config master: Bring publish-wheel-mirror-ubuntu-bionic job online https://review.openstack.org/550494 | 17:39 |
| *** myoung|ruck is now known as myoung|lunch | 17:40 | |
| *** andreas_s has quit IRC | 17:41 | |
| *** jpena is now known as jpena|off | 17:42 | |
| clarkb | mordred: re the emit header thing, can we update it to do the non nodepool content on localhost so it prints once then have a separate task that runs on all that will run the nodepool info? | 17:44 |
| andreaf | mordred when will I be able to stop fetching the logs from the devstack job? https://review.openstack.org/#/c/550191/ | 17:44 |
| clarkb | tobiash: ^ maybe that is what you were suggesting too? | 17:44 |
| *** andreas_s has joined #openstack-infra | 17:44 | |
| andreaf | mordred do I have to do something specific in devstack jobs to trigger logs fetching in base jobs? | 17:45 |
| tobiash | clarkb: nodepool provider is the same for all nodes | 17:45 |
| clarkb | tobiash: ya thats a good point (at least for current nodepool) | 17:46 |
| tobiash | clarkb: so currently one could just run on all and set 'once: True' | 17:46 |
| clarkb | tobiash: and once: True means just run one a single node that matches all? | 17:46 |
| tobiash | then it runs on a node where all vars are there and just once | 17:46 |
| tobiash | yes | 17:46 |
| tobiash | so it will run on a random (or first?) node that matches all | 17:47 |
| *** danpawlik has joined #openstack-infra | 17:47 | |
| *** lucasagomes is now known as lucas-afk | 17:47 | |
| *** armaan has quit IRC | 17:47 | |
| tobiash | clarkb: oh, in fact it's run_once | 17:48 |
| tobiash | http://docs.ansible.com/ansible/latest/playbooks_delegation.html#run-once | 17:48 |
| *** armaan has joined #openstack-infra | 17:48 | |
| *** gyee has joined #openstack-infra | 17:48 | |
| tobiash | we already use that in some places where we delegate to localhost for directory creation etc | 17:48 |
| tobiash | ok, the docs say it will pick the first host | 17:49 |
| *** felipemonteiro has joined #openstack-infra | 17:49 | |
| *** danpawlik has quit IRC | 17:52 | |
| *** camunoz has joined #openstack-infra | 17:53 | |
| *** armaan has quit IRC | 17:53 | |
| pabelanger | clarkb: I'm going to try and kick off a periodic job for openstack/requirements to confirm bionic job publishes correctly to AFS | 17:54 |
| *** tedlit has joined #openstack-infra | 17:54 | |
| pabelanger | any objections? | 17:54 |
| clarkb | pabelanger: not from me | 17:54 |
| pabelanger | ack | 17:54 |
| mordred | clarkb: wow, slack got rid of their IRC gateway? that was the only thing that made slack even *partially* usable rather than just being a completely crippled proprietary lock-in garbage pile | 17:54 |
| *** armaan has joined #openstack-infra | 17:54 | |
| clarkb | mordred: well in the process of getting rid of it ya | 17:54 |
| mordred | clarkb, tobiash: I like the run_once idea | 17:54 |
| clarkb | mordred: may 15 is day of no more slack irc gateway | 17:55 |
| mordred | clarkb: I didn't think it was possible for me to hate them more than I already did | 17:55 |
| tobiash | mordred, clarkb: does the nodepool var also has node specific things? | 17:55 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/infra-specs master: Add a spec to improve IRC discoverability https://review.openstack.org/550550 | 17:55 |
| pabelanger | okay, periodic jobs enqueued properly | 17:55 |
| dmsimard | infra-root: ^ an idea I got last night that I wrote down | 17:55 |
| mordred | tobiash: yes - but I don't think we were planning on printing any of it | 17:56 |
| clarkb | tobiash: I think it has everything in the nodeset info for that node | 17:56 |
| mordred | tobiash: it has ip addresses and label as well | 17:56 |
| mordred | which would be different per-node | 17:56 |
| tobiash | is that interesting to print? | 17:57 |
| *** pcaruana has joined #openstack-infra | 17:57 | |
| mordred | I don't think so? | 17:57 |
| *** andreas_s has quit IRC | 17:57 | |
| mordred | andreaf: not yet - need to finish the base job work | 17:57 |
| *** ykarel_ has joined #openstack-infra | 17:58 | |
| mordred | andreaf: and sorry - got distracted by other things - as soon as I've got the zuul-web issue sorted, I'll shift back to base job log fetching | 17:58 |
| tobiash | k, because in case we would want to print per node things then clarkb's idea to split nodepool info into its own task would make sense | 17:58 |
| clarkb | the ip addrs is actually something that would be helpful when holding nodes | 17:58 |
| clarkb | noticed this when helping dirk last week | 17:58 |
| clarkb | we had to wait for the firewall updates for multinode to happen to see what the ip addresses were | 17:58 |
| pabelanger | +1 | 17:58 |
| *** wolverineav has joined #openstack-infra | 17:58 | |
| *** derekh has quit IRC | 18:00 | |
| *** agopi has quit IRC | 18:00 | |
| *** spiffxp has quit IRC | 18:01 | |
| andreaf | mordred: ok - I was thinking about changing that role than to remove the extra "logs" folder - the current setup with inventory_hostname/logs/<log-files> is less then ideal | 18:02 |
| *** andreas_s has joined #openstack-infra | 18:02 | |
| *** vipul has quit IRC | 18:02 | |
| jlvillal | Review request openstack-infra/puppet-gerritbot: Have gerritbot install from git: https://review.openstack.org/#/c/546700/ Has one +2 already. Thanks :) | 18:02 |
| AJaeger | clarkb: could you review this for gitdm, please? https://review.openstack.org/549033 | 18:05 |
| clarkb | AJaeger: done | 18:06 |
| AJaeger | thanks, clarkb | 18:06 |
| * clarkb is going to afk now for "dinner" yay jetlag | 18:07 | |
| jlvillal | clarkb, You can still have "supper" later ;) | 18:07 |
| *** trown|rover is now known as trown|lunch | 18:08 | |
| clarkb | jlvillal: ya I'm going to hvae to make a better effort of transitioning back to local time today | 18:08 |
| clarkb | but I'm hungry now :) | 18:08 |
| jlvillal | :) | 18:08 |
| jlvillal | Going the other direction always messes me up. Coming back seems to be easier for me. | 18:08 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove gitdm legacy jobs https://review.openstack.org/550553 | 18:08 |
| *** yamamoto has joined #openstack-infra | 18:09 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove gitdm legacy jobs https://review.openstack.org/550554 | 18:09 |
| AJaeger | and follows for config-core ^ | 18:09 |
| *** cshastri has quit IRC | 18:10 | |
| *** hemna_ has joined #openstack-infra | 18:13 | |
| *** yamamoto has quit IRC | 18:13 | |
| *** VW__ has quit IRC | 18:14 | |
| *** agopi has joined #openstack-infra | 18:14 | |
| *** ralonsoh has quit IRC | 18:14 | |
| *** VW has joined #openstack-infra | 18:14 | |
| fungi | clarkb: pabelanger: a lingering todo with the wheel builder... we should compare the filenames against pypi (or a pypi mirror) and only publish ones which aren't already on pypi. right now our wheel mirrors contain a lot of stuff downloaded from pypi (but then fail to get deleted when the wheels on pypi get deleted) | 18:15 |
| *** VW has quit IRC | 18:15 | |
| *** pblaho has quit IRC | 18:15 | |
| *** VW has joined #openstack-infra | 18:16 | |
| *** rloo has joined #openstack-infra | 18:16 | |
| *** andreas_s has quit IRC | 18:16 | |
| rloo | Hi, I'm removing 'ironic-inspector-tempest-plugin' project since it was never used by ironic (https://review.openstack.org/550537). my question is, can we (you?) just delete the git repo since there isn't anything there except the .gitreview file? | 18:17 |
| *** danpawlik has joined #openstack-infra | 18:18 | |
| fungi | rloo: short answer, "no" | 18:18 |
| *** ykarel_ has quit IRC | 18:18 | |
| rloo | fungi: ok, short and sweet :) thx! | 18:19 |
| fungi | rloo: luckily, if you add a readme saying it's unused and follow the retirement process, its presence hopefully shouldn't confuse anyone | 18:19 |
| rloo | fungi: will do :) | 18:19 |
| *** suhdood has joined #openstack-infra | 18:19 | |
| *** slaweq has quit IRC | 18:20 | |
| fungi | rloo: https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project in case you need it | 18:20 |
| fungi | hopefully most of that is a no-op | 18:20 |
| rloo | fungi: yup, that's what i was following. just didn't want to put a readme for a repo that did nothing, but i can do it! | 18:20 |
| fungi | think of it more as a checklist | 18:20 |
| *** VW has quit IRC | 18:20 | |
| rloo | fungi: thx! | 18:21 |
| pabelanger | fungi: yah, that is a little easier to test now via depends-on, but have never looked more into it. Right now it takes about 45mins to get thought builds for each distro / python | 18:21 |
| fungi | rloo: the longer answer is that gerrit doesn't have a project deletion feature, so better to just mark it clearly as unused/abandoned/retired/whatever | 18:21 |
| *** david-lyle has joined #openstack-infra | 18:21 | |
| *** andreas_s has joined #openstack-infra | 18:22 | |
| rloo | fungi: ahh. ok, no worries. | 18:22 |
| *** danpawlik has quit IRC | 18:22 | |
| *** e0ne has quit IRC | 18:24 | |
| fungi | clarkb: do you recall specifically what part of pip 10 is going to break us? are you aware of any open gh isshe against pypa/pip for that? | 18:24 |
| *** dhajare has joined #openstack-infra | 18:24 | |
| pabelanger | mgagne: seeing some errors in inap right now, http://grafana.openstack.org/dashboard/db/nodepool-inap FYI | 18:24 |
| fungi | s/isshe/issue/ | 18:24 |
| mgagne | pabelanger: checking | 18:25 |
| pabelanger | I don't see any specific errors in nodepool, just network timeouts | 18:26 |
| fungi | clarkb: something about distutils-based packages no longer being able to shadow distro packages when installed into system context, right? | 18:26 |
| mgagne | pabelanger: network as pure networking or neutron? | 18:26 |
| *** slaweq has joined #openstack-infra | 18:27 | |
| *** gfidente is now known as gfidente|afk | 18:27 | |
| pabelanger | mgagne: no, mostly | 18:27 |
| pabelanger | nodepool.exceptions.ServerDeleteException: Timeout waiting for server 0d5eacff-b20a-466c-a582-1b75393fefb5 deletion | 18:27 |
| *** slaweq has quit IRC | 18:27 | |
| mgagne | hmm | 18:27 |
| pabelanger | or same with launch | 18:27 |
| *** olivierbourdon3- has quit IRC | 18:27 | |
| *** olivierbourdon38 has joined #openstack-infra | 18:28 | |
| *** slaweq has joined #openstack-infra | 18:28 | |
| clarkb | fungi: aiui the problem is pip 10 won't uninstall bin paths and other global paths if the original package was distutils based and many of the distro pacckages are distutils based | 18:29 |
| clarkb | fungi: and yes ianw opened a bug for it | 18:29 |
| clarkb | I'd have to go dig up the bug though, don't have the number handy | 18:29 |
| *** panda is now known as panda|off | 18:29 | |
| fungi | i'll find it | 18:30 |
| fungi | thanks! | 18:30 |
| *** olivierbourdon38 has quit IRC | 18:30 | |
| clarkb | fungi: and I have an example job log somewhere that shwos it in action, I'll get you a link for that | 18:30 |
| *** olivierbourdon38 has joined #openstack-infra | 18:31 | |
| *** shardy has quit IRC | 18:31 | |
| *** olivierbourdon38 has quit IRC | 18:32 | |
| *** olivierbourdon38 has joined #openstack-infra | 18:32 | |
| mgagne | pabelanger: will have to wait for coworkers to be available, root cause isn't obvious so far | 18:33 |
| *** neiljerram has quit IRC | 18:33 | |
| clarkb | fungi: http://logs.openstack.org/25/513825/6/check/tempest-full/ec9c70b/job-output.txt.gz#_2018-02-22_18_34_18_685381 | 18:33 |
| *** r-daneel has quit IRC | 18:33 | |
| fungi | thanks again! | 18:34 |
| clarkb | fungi: in that case psutil was already installed by apt due to devstack's system package installation then pip10 can't remove the files in eg /usr/bin aiui | 18:34 |
| *** r-daneel has joined #openstack-infra | 18:34 | |
| *** david-lyle has quit IRC | 18:35 | |
| *** andreas_s has quit IRC | 18:36 | |
| pabelanger | mgagne: np, just wanted to give you a heads up | 18:37 |
| jlvillal | clarkb, What is "pip10"? | 18:39 |
| openstackgerrit | Merged openstack-infra/gitdm master: Import Zuul v3 job https://review.openstack.org/549033 | 18:39 |
| clarkb | jlvillal: next release of pip, we are currently at pip version 9.something | 18:39 |
| jlvillal | clarkb, Ah. I thought it was a command "pip10" :) | 18:40 |
| *** andreas_s has joined #openstack-infra | 18:41 | |
| openstackgerrit | Ruby Loo proposed openstack-infra/project-config master: Remove ironic-inspector-tempest-plugin https://review.openstack.org/550537 | 18:43 |
| *** agopi has quit IRC | 18:47 | |
| *** efoley has quit IRC | 18:48 | |
| *** dtantsur is now known as dtantsur|afk | 18:48 | |
| clarkb | jlvillal: I just fail at typing :) | 18:48 |
| *** myoung|lunch is now known as myoung|ruck | 18:49 | |
| *** bobh has quit IRC | 18:51 | |
| *** VW has joined #openstack-infra | 18:52 | |
| *** tesseract has quit IRC | 18:53 | |
| *** danpawlik has joined #openstack-infra | 18:56 | |
| *** dbecker has joined #openstack-infra | 18:57 | |
| *** VW has quit IRC | 18:58 | |
| *** sdake has joined #openstack-infra | 18:59 | |
| *** sdake has quit IRC | 18:59 | |
| *** sdake has joined #openstack-infra | 18:59 | |
| mgagne | pabelanger: we might have found the issue, currently being investigated | 18:59 |
| pabelanger | mgagne: yay | 18:59 |
| *** andreas_s has quit IRC | 18:59 | |
| mgagne | pabelanger: I see a lot of instances in "deleting" state, I think we had that issue in the past, I don't remember well how to get out of that state, will have to check | 19:00 |
| *** danpawlik has quit IRC | 19:00 | |
| *** david-lyle has joined #openstack-infra | 19:03 | |
| *** VW has joined #openstack-infra | 19:03 | |
| *** bobh has joined #openstack-infra | 19:04 | |
| AJaeger | mordred: what's failing with requirements? http://logs.openstack.org/75/550475/1/check/openstack-tox-py27/81349e2/job-output.txt.gz#_2018-03-07_17_38_11_344619 | 19:05 |
| AJaeger | mordred: regarding https://review.openstack.org/#/c/531825/ - should I create those missing pypi repos? | 19:06 |
| AJaeger | mordred: or ask teams to do it? | 19:07 |
| pabelanger | mgagne: not sure, I know we'll retry the delete but want to say there is also something on openstack side that needed to be done | 19:08 |
| pabelanger | something with setting the state | 19:08 |
| mgagne | pabelanger: yea, maybe task_state | 19:08 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/storyboard-webclient master: Update Private Label https://review.openstack.org/550572 | 19:08 |
| mgagne | pabelanger: will try to see what I can do, currently a bit overwhelm =) | 19:09 |
| *** olivierbourdon38 has quit IRC | 19:09 | |
| *** yamamoto has joined #openstack-infra | 19:10 | |
| *** trown|lunch is now known as trown|rover | 19:10 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:10 | |
| pabelanger | mgagne: np! we'll manage over here :) | 19:10 |
| *** eernst has quit IRC | 19:10 | |
| *** olivierbourdon38 has quit IRC | 19:13 | |
| *** yamamoto has quit IRC | 19:15 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:16 | |
| *** olivierbourdon38 has quit IRC | 19:18 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:19 | |
| *** mgoddard_ has joined #openstack-infra | 19:20 | |
| mgoddard_ | hi infra team, is anyone able to review https://review.openstack.org/#/c/546696? Sorry in advance if there is magic going on behind the scenes and I'm just being impatient... | 19:22 |
| *** erlon has quit IRC | 19:24 | |
| clarkb | mgoddard_: I think for most of us its recovery from travel as biggest hold up on $reviews. I've approved it | 19:24 |
| *** harlowja has joined #openstack-infra | 19:26 | |
| mgoddard_ | clarkb: of course, I'm based in the UK so didn't have too far to go | 19:26 |
| mgoddard_ | clarkb: thanks | 19:26 |
| *** danpawlik has joined #openstack-infra | 19:27 | |
| mgoddard_ | clarkb: so at this point I believe I need to request to be added to the kayobe-core and kayobe-release gerrit groups. Is that something you could help with? | 19:27 |
| fungi | pabelanger: were you still planning to spin up a review01.openstack.org or whatever running xenial sometime this week so we could send an announcement about the ip address change as soon as possible? | 19:29 |
| clarkb | mgoddard_: yes let me take a look | 19:29 |
| *** olivierb_ has joined #openstack-infra | 19:29 | |
| *** dougwig has joined #openstack-infra | 19:29 | |
| clarkb | mgoddard_: done | 19:31 |
| *** danpawlik has quit IRC | 19:31 | |
| mgoddard_ | clarkb: excellent, thanks for your help - that will allow us to get started using gerrit | 19:32 |
| openstackgerrit | Ruby Loo proposed openstack-infra/project-config master: Retire ironic-inspector-tempest-plugin https://review.openstack.org/550537 | 19:32 |
| pabelanger | fungi: yah, I was hoping to do review-dev.o.o first tomorrow, just to confirm there isn't any puppet changes needed | 19:32 |
| openstackgerrit | Merged openstack-infra/project-config master: Add Zuul project for kayobe https://review.openstack.org/546696 | 19:32 |
| *** r-daneel_ has joined #openstack-infra | 19:34 | |
| *** r-daneel has quit IRC | 19:35 | |
| *** r-daneel_ is now known as r-daneel | 19:35 | |
| *** olivierbourdon38 has quit IRC | 19:36 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:36 | |
| *** olivierbourdon38 has quit IRC | 19:38 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:39 | |
| *** nicolasbock has joined #openstack-infra | 19:39 | |
| openstackgerrit | Emmet Hikory proposed openstack-infra/project-config master: Bring wheel builder for ubuntu-xenial-arm64 online https://review.openstack.org/549644 | 19:40 |
| fungi | pabelanger: cool. i also need to do some gerrit its-storyboard plugin testing between review-dev and storyboard-dev but we can coordinate | 19:42 |
| *** olivierbourdon38 has quit IRC | 19:43 | |
| *** e0ne has joined #openstack-infra | 19:43 | |
| ianw | pabelanger / persia: ubuntu ports mirroring still going ... | 19:43 |
| *** olivierbourdon38 has joined #openstack-infra | 19:43 | |
| mgagne | pabelanger: ok, need to update task_state to NULL and somehow either nodepool or openstack fixes itself | 19:43 |
| pabelanger | fungi: yah, i think if puppet applies cleanly to review-dev, we can then see about bringing online review01.o.o server to obtain ip | 19:44 |
| AJaeger | mordred: won't have time to do further work on those tox_install stuff today - I'm just updating etherpad with some comments | 19:44 |
| pabelanger | ianw: yah, I expect it to take a while | 19:44 |
| pabelanger | mgagne: ack, grafana looks better now | 19:44 |
| openstackgerrit | Emmet Hikory proposed openstack-infra/project-config master: Publish Ubuntu Xenial ARM64 wheels https://review.openstack.org/550582 | 19:44 |
| ianw | i've woken up to no internet ... clearly a request such as "enable location A on day B and disable location C on day D is far too difficult | 19:44 |
| clarkb | ianw: you'll be happy to know wesuffer similar problems on this side of the pacific too | 19:44 |
| clarkb | just ask calebb | 19:44 |
| mtreinish | fungi: well, so far no crashes on firehose.o.o since the update to 1.4.15 which is a good sign | 19:45 |
| * mtreinish is actively trying to jinx it as a reliability test | 19:45 | |
| fungi | ianw: back when i worked on data centers, circuit disconnect orders were always a problem. by policy we stopped putting in disconnect orders in advance because providers would get a disconnect order for a month out and would just process it immediately anyway | 19:45 |
| persia | ianw: Wow! Somehow I thought it would only take 12-15 hours. | 19:45 |
| clarkb | ianw: fwiw gerrit continues to look happy after your surgery on nova-specs | 19:46 |
| clarkb | ianw: I think we might be able to call that fixed now | 19:46 |
| ianw | the new apartment has fibre ... i'm hopeful i can just plug my unifi gateway into it, because they haven't managed to get me their modem | 19:46 |
| ianw | \o/ yay! that one took a while | 19:46 |
| bkero- | ianw: is it centurylink? | 19:46 |
| *** suhdood has quit IRC | 19:47 | |
| *** olivierbourdon38 has quit IRC | 19:47 | |
| fungi | centurylink does business in au? | 19:47 |
| *** dbecker has quit IRC | 19:47 | |
| bkero- | Hah maybe not then | 19:47 |
| ianw | bkero-: it's what we call "NBN" (national broadband network) in .au ... which is a whole big thing | 19:47 |
| bkero- | I have centurylink fiber with a ubnt edgerouter, it's on vlan221 with pppoe | 19:47 |
| bkero- | I imagine yours will be very different :) | 19:47 |
| clarkb | since we are on the subject my water company is doing work on our street which should not affect power/internets but they do have a backhoe on site and I think I hear a cement saw. Here's hoping they don't screw anything up :) | 19:48 |
| *** olivierbourdon38 has joined #openstack-infra | 19:48 | |
| fungi | for one thing, all the bits flip when they cross the equator so you need entirely different equipment in that hemisphere | 19:48 |
| ianw | bkero-: interesting ... this is a new build; not many people have fibre to premises, and i can't find much discussion on what the grey box with 4 ethernet ports on the wall actually is | 19:49 |
| fungi | clarkb: teh sidewalk they've been working on installing on my street for months now has resulted in at least half a dozen lengthy network outages for me, and they're nowhere near done yet either. fiber-seeking backhoes are everywhere | 19:49 |
| bkero- | ianw: I hope it ends up being fairly straightforward | 19:50 |
| clarkb | fungi: they did get all the utilities marked so they started properly at least | 19:50 |
| fungi | same here | 19:50 |
| fungi | didn't seem to help | 19:50 |
| fungi | or maybe it did help, and would have been an order of magnitude worse otherwise | 19:50 |
| *** olivierb_ has quit IRC | 19:51 | |
| *** olivierbourdon38 has quit IRC | 19:51 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:51 | |
| *** tedlit_ has joined #openstack-infra | 19:51 | |
| *** VW_ has joined #openstack-infra | 19:52 | |
| *** ihrachys_ has joined #openstack-infra | 19:52 | |
| *** ihrachys has quit IRC | 19:52 | |
| *** germs_ has joined #openstack-infra | 19:52 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Store label info with launcher registration https://review.openstack.org/548376 | 19:52 |
| *** olivierbourdon38 has quit IRC | 19:53 | |
| *** olivierbourdon38 has joined #openstack-infra | 19:53 | |
| *** germs__ has joined #openstack-infra | 19:54 | |
| *** VW has quit IRC | 19:54 | |
| *** tedlit has quit IRC | 19:54 | |
| *** eharney has quit IRC | 19:54 | |
| *** germs has quit IRC | 19:54 | |
| *** jamesdenton has quit IRC | 19:54 | |
| *** mgoddard_ has quit IRC | 19:55 | |
| *** germs_ has quit IRC | 19:57 | |
| *** germs__ is now known as germs | 19:58 | |
| *** germs has quit IRC | 19:58 | |
| *** germs has joined #openstack-infra | 19:58 | |
| *** felipemonteiro has quit IRC | 19:58 | |
| *** eharney has joined #openstack-infra | 19:59 | |
| *** jamesmcarthur has joined #openstack-infra | 19:59 | |
| *** danpawlik has joined #openstack-infra | 20:01 | |
| *** bobh has quit IRC | 20:01 | |
| pabelanger | mirror.ubuntu has staled again, we lost AFS during reprepro last night. This is about the 5 time in 2 weeks it has happened. We might be at the limits of amount of distros we are watching on the mirror.ubuntu volume | 20:02 |
| pabelanger | we might have to refactor some how, to sync to disk first, then rsync to afs? | 20:02 |
| clarkb | its only one distro right? releases you mean? | 20:02 |
| pabelanger | yah, releases. | 20:02 |
| *** dhajare has quit IRC | 20:02 | |
| pabelanger | ubuntu trusty / xenial / bionic | 20:02 |
| clarkb | pabelanger: could also be that doing bionic + arm64 at the same time has it unhappy? | 20:03 |
| pabelanger | yah, that is also possible | 20:03 |
| pabelanger | maybe we are doing too much network IO from that server | 20:03 |
| pabelanger | to afs cell | 20:03 |
| fungi | check cacti graphs for network interfaces of the servers involved and see if they look like they're getting capped hard. that could imply significant packet loss during those timeframes | 20:05 |
| *** danpawlik has quit IRC | 20:06 | |
| *** david-lyle has quit IRC | 20:07 | |
| pabelanger | yah, could be, I see a lot of traffic yesterday | 20:08 |
| pabelanger | which is likely related to work ianw is doing for arm64 | 20:08 |
| *** r-daneel_ has joined #openstack-infra | 20:09 | |
| *** r-daneel has quit IRC | 20:09 | |
| *** r-daneel_ is now known as r-daneel | 20:09 | |
| *** jamesmcarthur has quit IRC | 20:10 | |
| *** jamesmcarthur has joined #openstack-infra | 20:11 | |
| *** yamamoto has joined #openstack-infra | 20:11 | |
| *** hemna_ has quit IRC | 20:13 | |
| *** gfidente|afk is now known as gfidente | 20:15 | |
| *** david-lyle has joined #openstack-infra | 20:15 | |
| *** yamamoto has quit IRC | 20:16 | |
| *** caphrim007 has quit IRC | 20:20 | |
| *** pblaho has joined #openstack-infra | 20:22 | |
| *** ihrachys_ has quit IRC | 20:24 | |
| *** jtomasek has quit IRC | 20:24 | |
| *** ihrachys has joined #openstack-infra | 20:24 | |
| *** VW has joined #openstack-infra | 20:24 | |
| *** trown|rover is now known as trown|brb | 20:25 | |
| *** ldnunes has quit IRC | 20:25 | |
| *** ldnunes has joined #openstack-infra | 20:25 | |
| openstackgerrit | Ildiko Vancsa proposed openstack-infra/system-config master: Add #edge-computing-group to hiera/common.yaml https://review.openstack.org/549201 | 20:27 |
| *** VW_ has quit IRC | 20:27 | |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Create new project for oslo.limit https://review.openstack.org/550491 | 20:28 |
| *** VW_ has joined #openstack-infra | 20:28 | |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Add oslo.limit to zuul project list https://review.openstack.org/550591 | 20:28 |
| ianw | pabelanger: see also https://review.openstack.org/#/c/540198/ | 20:28 |
| ianw | auristor has made these suggestions of things we could do better. it might help | 20:29 |
| *** VW has quit IRC | 20:29 | |
| openstackgerrit | Ildiko Vancsa proposed openstack-infra/system-config master: Add #edge-computing-group to hiera/common.yaml https://review.openstack.org/549201 | 20:29 |
| ianw | note the ports mirror is on a different volume, mirror.ubuntu-ports too | 20:30 |
| pabelanger | wfm | 20:31 |
| *** jamesmcarthur has quit IRC | 20:31 | |
| *** trown|brb is now known as trown|rover | 20:31 | |
| ianw | yeah, updating the settings is a bit of a pain, i think you have to re-install it? | 20:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/infra-specs master: Add survey spec https://review.openstack.org/349831 | 20:34 |
| clarkb | finally doing some spec cleanup ^ reabsed and added option for using ethercalc for simple surveys | 20:34 |
| *** kjackal has joined #openstack-infra | 20:36 | |
| *** eernst has joined #openstack-infra | 20:37 | |
| *** danpawlik has joined #openstack-infra | 20:38 | |
| *** d0ugal has quit IRC | 20:40 | |
| *** danpawlik has quit IRC | 20:43 | |
| persia | Regarding AFS mirrors: `ls /afs/openstack.org/mirror` generates "ls: cannot access '/afs/openstack.org/mirror/npm': No such device" : if things are being cleaned, that might be another useful target. | 20:43 |
| pabelanger | yah, we likey need to delete that | 20:44 |
| pabelanger | we removed mirror/npm a while back | 20:44 |
| mordred | on some level I think you could tell the entire story of openstack purely in terms of what our mirror and dependency management story was at a given point in time | 20:45 |
| mordred | I'm guessing almost nobody remembers when we were attempting to keep cached pre-built virtualenvs and then to copy/relocate them somewhere else | 20:46 |
| *** tedlit has joined #openstack-infra | 20:46 | |
| mordred | I think clarkb might have been around for the tail end of that one | 20:46 |
| clarkb | seems like I remember learning venvs are not relocatable or portable across python releases | 20:47 |
| mordred | yah. they get really angry when you try to relocate them | 20:47 |
| tobiash | forcefully relocation is rude ;) | 20:48 |
| mordred | tobiash: :) | 20:48 |
| tobiash | we should have learned that by history ;) | 20:49 |
| *** tedlit_ has quit IRC | 20:49 | |
| *** d0ugal has joined #openstack-infra | 20:52 | |
| pabelanger | mordred: isn't that what giftwrap was doing? installing packages into virtualenvs and copypasta them into same arch nodes? | 20:52 |
| *** david-lyle has quit IRC | 20:53 | |
| mordred | pabelanger: yup. technology changed over the course of a few years :) | 20:54 |
| *** tedlit has quit IRC | 20:54 | |
| *** tedlit has joined #openstack-infra | 20:54 | |
| pabelanger | mordred: yah, I never got around to trying giftwrap, but thought it was working. As long as builds happened on same distro / python versions | 20:55 |
| clarkb | I'm not entirely sure its still a good/safe way to do it as python updates would still break it iirc | 20:55 |
| pabelanger | yah, unsure | 20:55 |
| fungi | clarkb: python updates do indeed break virtualenvs | 20:56 |
| fungi | every time i upgrade python i have to rebuild my virtualenvs | 20:56 |
| * mordred has started using pyenv on his laptop for python fwiw | 20:56 | |
| mordred | pretty happy with it so far | 20:56 |
| mordred | it lets you add whichever python versions - and then to set default versions both globally and on a per-directory basis | 20:57 |
| mordred | it's also just actual pure upstream python | 20:57 |
| *** gouthamr has quit IRC | 20:57 | |
| *** ihrachys has quit IRC | 20:57 | |
| mordred | but after my upgrade, however premature if may have been, to bionic - it's how I got python3.5 for tox' | 20:58 |
| clarkb | except pure upstream python wont build on half the distros :( | 20:58 |
| mordred | pyenv has done a fine job with it on ubuntu bionic | 20:58 |
| clarkb | ya for paclage side patching instead of upstream fixes | 20:58 |
| clarkb | mordred: I think suse and red hat are more likely to break | 20:58 |
| clarkb | debian seems to do a good job | 20:58 |
| persia | There was a time that this was the best way to do things, That time may have passed. | 20:59 |
| *** ihrachys has joined #openstack-infra | 20:59 | |
| * mordred has 2.7.14 and 3.5.5 installed now | 20:59 | |
| persia | There are still packages in Debian that require two visits to a specific bar in order to get a new upstream release, which will be delivered on floppy. | 20:59 |
| mordred | clarkb: now you have me curious as to whether pyenv works or doesn't on rh and suse ... | 21:00 |
| persia | Most upstreams have more flexible communications policies. | 21:00 |
| clarkb | mordred: pretty sure it wont on suse based on me trying to build vanilla opemstack on suse | 21:00 |
| *** hashar has quit IRC | 21:00 | |
| clarkb | and I want to say similar issues with lib locations are present on centos | 21:00 |
| mordred | clarkb: we have tumbleweed nodes now right? | 21:01 |
| clarkb | mordred: ya | 21:02 |
| clarkb | er build vanilla python wee jetlag | 21:02 |
| *** ykarel has joined #openstack-infra | 21:03 | |
| fungi | clarkb: i build pure upstream source for latest 2.7 and 3.3-3.7 on my systems. granted i maintain a cheatsheet of which build commands and options to use for each | 21:10 |
| clarkb | the problems I ran intonwas distros seemed to notnot upstream lib names for things like curses | 21:11 |
| mordred | ok: https://review.openstack.org/550606 DNM testing building some things on some nodes | 21:11 |
| *** caphrim007 has joined #openstack-infra | 21:11 | |
| mordred | there's me cheating and using the gate to find out how well it works | 21:11 |
| *** pcaruana has quit IRC | 21:12 | |
| fungi | clarkb: http://paste.openstack.org/show/693824 is my list of build notes | 21:12 |
| *** HeOS has joined #openstack-infra | 21:12 | |
| *** yamamoto has joined #openstack-infra | 21:12 | |
| ianw | ok, i've got a trunk full of ethernet cables, a broadband cpe in an unknown state of activation and a head full of dreams. wish me luck ... ttyl | 21:13 |
| fungi | maybe harry tuttle will come to your rescue | 21:13 |
| fungi | granted he dealt more with ducts than cables | 21:14 |
| *** danpawlik has joined #openstack-infra | 21:14 | |
| mordred | fungi: you might find https://github.com/pyenv/pyenv interesting if you haven't looked at it yet - it's all just a bunch of shell | 21:14 |
| fungi | mordred: does it handle the source builds for you? | 21:15 |
| mordred | fungi: yup | 21:15 |
| *** ihrachys has quit IRC | 21:15 | |
| *** ihrachys has joined #openstack-infra | 21:15 | |
| *** andreas_s has joined #openstack-infra | 21:15 | |
| mordred | you basically say "pyenv install 3.5.5" and it grabs 3.5.5 from upstream, builds it and sticks it somewhere so you can request to use it later | 21:16 |
| mordred | it's .... way more reasonable than I was expecting | 21:16 |
| fungi | cool, i wasn't picking that up from the readme | 21:16 |
| *** VW_ has quit IRC | 21:17 | |
| *** yamamoto has quit IRC | 21:17 | |
| *** VW has joined #openstack-infra | 21:17 | |
| fungi | looks like a lot of magic that could set off my paranoia. so far i've been perfectly happy just keeping symlinks in ~/bin to my interpreters | 21:18 |
| *** VW has quit IRC | 21:18 | |
| *** danpawlik has quit IRC | 21:19 | |
| mordred | fungi: yah - I hear that ... the build itself is actually done through a different thing (that can apparently be standalone) https://github.com/pyenv/pyenv/tree/master/plugins/python-build | 21:19 |
| fungi | and i also never use virtualenv's `activate` script to begin with (symlinks have worked fine for that too) | 21:19 |
| *** tosky has quit IRC | 21:19 | |
| *** VW has joined #openstack-infra | 21:19 | |
| fungi | but i guess not having to rebuild my virtualenvs when i update my interpreters would be swell | 21:20 |
| *** andreas_s has quit IRC | 21:20 | |
| dhellmann | zuul experts: I'm working with the requirements team on a plan to add unit test jobs to projects currently syncing their requirements from the global list. | 21:20 |
| mordred | fungi: which is the thing that,if I'm reading it all correctly, is where they keep any information about things needed for building things | 21:20 |
| dhellmann | the test jobs would be defined in-repo most likely, as with https://review.openstack.org/#/c/550603/ | 21:20 |
| dhellmann | because we can't turn the job on for every repo all at once in case it breaks for a project | 21:21 |
| mordred | dhellmann: neat! | 21:21 |
| dhellmann | how bad would it be to have 300+ patches like that one proposed all at once? (like if I script this, and submit a patch to each repo) | 21:21 |
| dhellmann | would zuul freak out over the number of configuration changes? | 21:21 |
| fungi | if all 300 contain zuul configuration, then potentially bad | 21:22 |
| dhellmann | i.e., should I batch the work, like corvus did with the name removal stuff? | 21:22 |
| dhellmann | yeah, they would | 21:22 |
| dhellmann | ok, so batching | 21:22 |
| *** r-daneel_ has joined #openstack-infra | 21:22 | |
| fungi | we've seen the current machine top out around 50 concurrent config changes (and that was pretty nasty performance-wise) | 21:22 |
| dhellmann | ok, I could do them in sets of 10 or something | 21:23 |
| fungi | yeah, 10 would probably be okay | 21:23 |
| dhellmann | that gives other folks space to have their own config changes of different sorts | 21:23 |
| *** r-daneel has quit IRC | 21:23 | |
| *** r-daneel_ is now known as r-daneel | 21:23 | |
| fungi | looks like we're probably about due for a zuul restart to free up used memory since it's now almost maxing out with buffers/cache http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=64792&rra_id=all | 21:25 |
| mordred | fungi: yes. that's actually on the todo-list for once I finish fixing zuul-web ... | 21:25 |
| mordred | fungi: but strictly speaking there's really nothing preventing doing it sooner | 21:26 |
| mordred | fungi: other than the fact that we're pinned back a few revisions in /opt/zuul on zuul01 right now | 21:26 |
| *** tosky has joined #openstack-infra | 21:27 | |
| clarkb | ya mostly just didnt want to impact your fixing of things I think | 21:27 |
| *** dprince has quit IRC | 21:27 | |
| * mordred is still poking at things on other nodes - so if someone wants to restart, it will not affect me | 21:28 | |
| mordred | and by poking, I mean I'm now circling back to this after having walked away muttering earlier | 21:28 |
| fungi | right, it looks like it can wait until we want to restart it to pick up some new commits anyway | 21:29 |
| clarkb | after being awake at like 3am I feel like my day may end up slowly petering out over the next couple hours | 21:30 |
| tobiash | mordred: just updated zuul in my testenv | 21:33 |
| tobiash | got zuul-web working | 21:33 |
| tobiash | but console log behaves strange | 21:33 |
| tobiash | it's black at first but every added line gets a white background | 21:34 |
| *** jzerebecki has quit IRC | 21:34 | |
| mordred | oh - that's weird | 21:34 |
| mordred | tobiash: I wonder if I missed importing a css file or something | 21:35 |
| mordred | tobiash: or - alternately, if it's now importing a css file it wasn't before so we need to change the order | 21:35 |
| tobiash | that sounds possible | 21:35 |
| mordred | tobiash: did you have to do anything strange other than just installing the javascript tools? | 21:36 |
| tobiash | I had to restructure my dockerfile... | 21:36 |
| tobiash | my initial install method didn't work | 21:36 |
| tobiash | then tried to switch to clone within the dockerfile and installing from pip | 21:37 |
| tobiash | just to notice that pip in alpine is broken with symlinks | 21:37 |
| tobiash | and now I'm using pip for installing the requirements and python setup.py to install zuul afterwards... | 21:38 |
| mordred | nod. | 21:38 |
| tobiash | then with yarn installed that worked | 21:38 |
| tobiash | from zuul-web I just had to remove the manual akwart cruft to get the deps | 21:39 |
| tobiash | and my queue size patch doesn't work :/ | 21:40 |
| tobiash | reports constantly 0 | 21:41 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Store label info with launcher registration https://review.openstack.org/548376 | 21:42 |
| tobiash | the json is correct but zuul-web reports 0 | 21:44 |
| *** danpawlik has joined #openstack-infra | 21:46 | |
| fungi | infra-root: i've approved a couple of meetbot additions just now, | 21:47 |
| fungi | but there's still one lingering which needs a second core review and would be good to get in at roughly the same time: https://review.openstack.org/522711 | 21:47 |
| clarkb | fungi: its safe right now | 21:48 |
| clarkb | ? | 21:48 |
| mordred | clarkb: https://review.openstack.org/#/c/550606/ <-- pyenv totally builds python fine on centos and suse | 21:49 |
| clarkb | huh | 21:50 |
| clarkb | I wonder if its just not building things like curses then | 21:50 |
| mordred | I had it do 2.7.14 and 3.5.5 - also with -v so the configure logs are avail | 21:51 |
| mordred | clarkb: | 21:51 |
| *** pblaho has quit IRC | 21:51 | |
| mordred | 2018-03-07 21:43:28.283180 | centos-7 | building '_curses' extension | 21:51 |
| fungi | clarkb: looked like it was a safe time, yes | 21:51 |
| openstackgerrit | Dirk Mueller proposed openstack-infra/project-config master: Drop kingbird, networking-hpe and trio2o from g-r sync https://review.openstack.org/550415 | 21:51 |
| *** danpawlik has quit IRC | 21:51 | |
| clarkb | ok approved | 21:51 |
| clarkb | mordred: and for suse? maybe they are vendoring all that stuff? in any case I will have to try it again | 21:52 |
| mordred | clarkb: in any case, whatever they're doing, they seem to be doing a decent job of it | 21:52 |
| clarkb | I thought I even tried it with pyenv after vanilla autoconf and make failed on me | 21:52 |
| mordred | clarkb: yah. http://logs.openstack.org/06/550606/3/check/shade-pyenv-suse/0534ec2/ | 21:52 |
| mordred | clarkb: that's on a tumbleweed node | 21:52 |
| mordred | https://review.openstack.org/#/c/550606/3/playbooks/suse.yaml <-- I had it install what the pyenv folks recommend to install for build depends | 21:53 |
| mordred | clarkb: oh - it did fail to build curses on suse | 21:54 |
| *** HeOS has quit IRC | 21:55 | |
| mordred | clarkb: https://bugs.python.org/issue12271 fwiw - apparently suse puts panel.h into /usr/include/ncurses but other things have it just in /usr/include | 21:56 |
| openstackgerrit | Merged openstack-infra/system-config master: Add #edge-computing-group to hiera/common.yaml https://review.openstack.org/549201 | 21:56 |
| *** ykarel has quit IRC | 21:57 | |
| clarkb | mordred: I love it when the bugs are that old for something relatively simple | 21:59 |
| mordred | clarkb: also - closed, not a bug | 21:59 |
| *** lukebrowning has joined #openstack-infra | 22:00 | |
| openstackgerrit | Merged openstack-infra/system-config master: Log and archive #openstack-loci https://review.openstack.org/549217 | 22:01 |
| * mnaser is curious about the meetbot 'safe time' change? | 22:03 | |
| mnaser | no soft reloads? | 22:03 |
| clarkb | mnaser: correct, its a full restart of the bot so if it happens during a meeting your meeting stops (at least from the bot's perspective) | 22:04 |
| *** HeOS has joined #openstack-infra | 22:05 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: added new endpoint add location banner https://review.openstack.org/550625 | 22:05 |
| mnaser | ah i see | 22:05 |
| *** lukebrowning has quit IRC | 22:06 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add openstack-masakari to meetbot and statusbot channels https://review.openstack.org/522711 | 22:07 |
| *** r-daneel has quit IRC | 22:07 | |
| *** david-lyle has joined #openstack-infra | 22:08 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: added new endpoint add location banner https://review.openstack.org/550625 | 22:08 |
| *** trown|rover is now known as trown|outtypewww | 22:08 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: added new endpoint add location banner https://review.openstack.org/550625 | 22:10 |
| *** HeOS has quit IRC | 22:11 | |
| *** openstackstatus has quit IRC | 22:13 | |
| *** openstack has joined #openstack-infra | 22:14 | |
| *** ChanServ sets mode: +o openstack | 22:14 | |
| *** david-lyle has quit IRC | 22:14 | |
| clarkb | dirk: dmsimard: any idea if that iptables thing has been turned into a kernel bug? | 22:18 |
| *** yamamoto has quit IRC | 22:18 | |
| *** danpawlik has quit IRC | 22:21 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: DNM testing python3 without changes https://review.openstack.org/550629 | 22:24 |
| ianw | hello world | 22:25 |
| *** dbecker has joined #openstack-infra | 22:26 | |
| clarkb | ianw: you have internets again? | 22:26 |
| mordred | clarkb: zomg. I finally have a node I can run puppet on | 22:27 |
| mordred | clarkb: it probably should not have taken me this long ... | 22:27 |
| *** gfidente has quit IRC | 22:27 | |
| clarkb | mordred: virt-install should mostly work on upstream $distro images | 22:28 |
| *** ihrachys_ has joined #openstack-infra | 22:28 | |
| clarkb | ianw has a blog post somewhere on making a config drive for that to work | 22:28 |
| clarkb | (I just have a single config drive that I made forever ago and the command in a file to use it) | 22:28 |
| *** ihrachys has quit IRC | 22:29 | |
| mordred | clarkb: it wasn't getting a vm that was the hard part - it was getting enough context in to it for the things to work | 22:29 |
| clarkb | ah | 22:29 |
| clarkb | I read finally have a node I can run puppet on to mean getting a vm | 22:29 |
| mordred | clarkb: without getting _too_ much context in to it so that it behaves poorly | 22:29 |
| clarkb | you mean that you can run puppet on successfully ? | 22:29 |
| mordred | clarkb: nah. booting vms is a thing I'm fairly good at :) | 22:29 |
| mordred | clarkb: yah. or, that it doesn't bomb out immediately due to lack of hiera values | 22:30 |
| *** ldnunes has quit IRC | 22:30 | |
| *** bobh has joined #openstack-infra | 22:31 | |
| *** dbecker has quit IRC | 22:31 | |
| *** VW_ has joined #openstack-infra | 22:32 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added new endpoint get banners per location https://review.openstack.org/550631 | 22:33 |
| *** dklyle has joined #openstack-infra | 22:34 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added new endpoint get banners per location https://review.openstack.org/550631 | 22:35 |
| *** esberglu has quit IRC | 22:35 | |
| *** VW has quit IRC | 22:36 | |
| *** eharney has quit IRC | 22:36 | |
| *** kjackal has quit IRC | 22:36 | |
| *** VW_ has quit IRC | 22:36 | |
| *** rcernin has joined #openstack-infra | 22:37 | |
| *** edmondsw has quit IRC | 22:38 | |
| *** edmondsw has joined #openstack-infra | 22:39 | |
| *** openstackstatus has quit IRC | 22:42 | |
| *** openstack has joined #openstack-infra | 22:43 | |
| *** ChanServ sets mode: +o openstack | 22:43 | |
| *** edmondsw has quit IRC | 22:44 | |
| dmsimard | clarkb: I started bisecting the iptables thing, haven't found the source yet. I'm not positive on kernel yet. | 22:45 |
| *** slaweq has quit IRC | 22:46 | |
| clarkb | dmsimard: fwiw I think fedora and suse have different userland firewall implementations so kernel seems likely but probably best ot bisect and be sure | 22:46 |
| *** slaweq has joined #openstack-infra | 22:46 | |
| *** tpsilva has quit IRC | 22:47 | |
| dmsimard | It started occuring during the lifetime of F26 for sure because it was working before so my work has been looking at changelogs for stuff like netfilter/iptables/kernel to see if anything stands out | 22:47 |
| dmsimard | There's been a couple kernel dot releases, it's fairly easy to just try them all and see what happens. | 22:48 |
| *** esberglu has joined #openstack-infra | 22:50 | |
| *** slaweq has quit IRC | 22:51 | |
| *** tedlit has quit IRC | 22:52 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Actually honor the static_path config value https://review.openstack.org/550636 | 22:53 |
| *** HeOS has joined #openstack-infra | 22:53 | |
| mordred | clarkb, dmsimard, pabelanger: ^^ if'n you don't mind a quick little review | 22:53 |
| *** david-lyle has joined #openstack-infra | 22:54 | |
| *** tedlit has joined #openstack-infra | 22:54 | |
| *** dklyle has quit IRC | 22:55 | |
| mordred | also - if you want to double-check - 104.239.149.123 monty-test-zuul-web.openstack.org is the server I have up and running that ran with the proposed puppet-zuul changes - and that change in place in the local python | 22:55 |
| mordred | ianw: ^^ you too | 22:56 |
| mordred | https://review.openstack.org/#/c/549608/ <-- is the puppet-zuul change | 22:56 |
| *** slaweq has joined #openstack-infra | 22:56 | |
| mordred | if we land both changes, I'm pretty confident that things will work and that we can remove things from the emergency file | 22:57 |
| clarkb | mordred: I traced static_path through the zuul web command to web __init__.py to your change so I'm good with it | 22:58 |
| *** armaan has quit IRC | 22:58 | |
| *** armaan has joined #openstack-infra | 22:59 | |
| ianw | cool, i thought you needed a documentroot maybe | 22:59 |
| clarkb | probably plan to run puppet manually again? | 23:00 |
| mordred | clarkb: yah. since it's still in emergency - I think land the two changes, then do a manual sync/run on the node just to make sure | 23:00 |
| mordred | ianw: yah. I was sure it was gonna be something with the apache config | 23:01 |
| *** slaweq has quit IRC | 23:01 | |
| *** slaweq has joined #openstack-infra | 23:06 | |
| ianw | ohh, ubuntu-ports mirroring is finishing up | 23:07 |
| *** esberglu has quit IRC | 23:09 | |
| *** myoung|ruck is now known as myoung|off | 23:11 | |
| *** slaweq has quit IRC | 23:11 | |
| *** threestrands has joined #openstack-infra | 23:12 | |
| *** threestrands has quit IRC | 23:12 | |
| *** threestrands has joined #openstack-infra | 23:12 | |
| *** dhill__ has quit IRC | 23:14 | |
| *** yamamoto has joined #openstack-infra | 23:15 | |
| mordred | infra-root: could I get a +A on https://review.openstack.org/#/c/549608/ - it's been tested on a real server - and is needed to remove zuul01 from the emergency file | 23:18 |
| *** yamamoto has quit IRC | 23:20 | |
| *** yamamoto has joined #openstack-infra | 23:20 | |
| eernst | clarkb: in case you have input: https://github.com/kata-containers/ci/issues/6 | 23:22 |
| clarkb | mordred: does zuulv3 not need the lxml stuff? | 23:22 |
| clarkb | mordred: we finally removed that dep from all the places? I am so happy | 23:23 |
| mordred | clarkb: yah - it does not need it | 23:23 |
| openstackgerrit | Merged openstack-infra/zuul master: Actually honor the static_path config value https://review.openstack.org/550636 | 23:24 |
| clarkb | eernst: thats actually a largely unsolved problem for us too since as is mentioned there you need consistent hardware. I think we've decided we wouldn't need baremetal as long as the hypervisor isn't being used for other tasks as well. | 23:25 |
| clarkb | eernst: it just came up at the ptg, let me link to the resulting mailing list thread | 23:25 |
| eernst | k, thanks. | 23:25 |
| clarkb | eernst: http://lists.openstack.org/pipermail/openstack-dev/2018-March/127972.html | 23:26 |
| eernst | so, the "tl;dr" summary (after reading it :)) is that there isn't a set solution right now, is that right? | 23:28 |
| eernst | that is, a set solution with the openstrack infrastructure? | 23:29 |
| clarkb | eernst: correct | 23:29 |
| eernst | We are using a couple of baremetal machines right now internally and may keep down this route atm. | 23:29 |
| clarkb | eernst: with the major issue for us being where do we get hardware that will stick around long enough | 23:29 |
| eernst | Thanks. | 23:29 |
| *** danpawlik has joined #openstack-infra | 23:30 | |
| fungi | basically our current donated resources are all virtual machines in (frequently oversubscribed) public cloud providers, so performance consistency across providers or even within a single provider or even on the same virtual machine from one minute to the next could vary significantly | 23:30 |
| *** HeOS has quit IRC | 23:30 | |
| *** claudiub has quit IRC | 23:30 | |
| fungi | and to be able to have comparative performance between runs we need an entirely different kind of donated server resource | 23:31 |
| fungi | luckily the framework we're using could likely consume this new kind of donation without any real modification, so it's more a matter of logistics than anything | 23:32 |
| *** armaan has quit IRC | 23:33 | |
| *** armaan has joined #openstack-infra | 23:34 | |
| *** threestrands_ has joined #openstack-infra | 23:34 | |
| *** danpawlik has quit IRC | 23:35 | |
| *** threestrands_ has quit IRC | 23:35 | |
| *** threestrands_ has joined #openstack-infra | 23:36 | |
| *** threestrands has quit IRC | 23:36 | |
| clarkb | mordred: left a couple comments on the puppet-zuul change will let you decide if you want to try and address any of that before merging | 23:37 |
| *** slaweq has joined #openstack-infra | 23:37 | |
| clarkb | mordred: I'm on the fence of whether or not its worthaddressing them | 23:37 |
| *** VW has joined #openstack-infra | 23:37 | |
| *** tedlit has quit IRC | 23:37 | |
| *** slaweq has quit IRC | 23:42 | |
| *** slaweq has joined #openstack-infra | 23:47 | |
| *** VW has quit IRC | 23:51 | |
| *** hongbin has quit IRC | 23:51 | |
| *** VW has joined #openstack-infra | 23:52 | |
| *** slaweq has quit IRC | 23:52 | |
| *** VW has quit IRC | 23:56 | |
| *** slaweq has joined #openstack-infra | 23:57 | |
| *** fifilip has joined #openstack-infra | 23:58 | |
| *** rlandy is now known as rlandy|bbl | 23:58 | |
| tonyb | mordred: given that shade provides the majority of the implementation for os_networks ... if we need new flags in os_networks we add it to shade and then send a PR against ansible/ansible to add the feature and bump the shade version from 1.6.0 ... #askingforafriend | 23:59 |
| clarkb | tonyb: and at this point it should be all CI'd by zuul too | 23:59 |
| tonyb | clarkb: Oh cool. | 23:59 |
| clarkb | tonyb: so you can add a test or something too maybe and check it all works at the end | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!