| mrodden | jog0: the git.openstack.org one is behind a commit | 00:00 |
|---|---|---|
| emagana | Infra masters! Have a question about my latest gerrit commit | 00:00 |
| emagana | The dependency is gone and this is why Jenkins is failing. | 00:00 |
| emagana | anyone could help me to add the dependency back? | 00:00 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add support for grenade-dsvm-rolling test https://review.openstack.org/64199 | 00:00 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add Trove testing support https://review.openstack.org/65040 | 00:01 |
| *** ociuhandu has quit IRC | 00:01 | |
| jeblair | jog0: i triggered replication and git.o.o is updated now | 00:01 |
| *** openstack has joined #openstack-infra | 00:02 | |
| *** mdenny has quit IRC | 00:02 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Ensure a recent(48h) check result https://review.openstack.org/69508 | 00:04 |
| jeblair | clarkb, mordred, fungi, sdague, mikal: ^ that's where we can vote on (b). :) | 00:04 |
| clarkb | emagana: git review -d thedependency then git review -d thefailingchange, then git rebase -i branchfordependency, git review | 00:05 |
| clarkb | mordred: right running manage-projects once I get through more review is on my list of things to do | 00:05 |
| clarkb | I approved at least 2 changes that need it to be run | 00:06 |
| emagana | clarkb: let me try that | 00:06 |
| openstackgerrit | A change was merged to openstack-infra/config: Switch whisper to master branch https://review.openstack.org/66189 | 00:07 |
| emagana | clarkb: what would be "thefailingchange"? | 00:07 |
| emagana | clarkb: My commit https://review.openstack.org/#/c/67312/ should depend on this one https://review.openstack.org/#/c/67210/ | 00:07 |
| clarkb | emagana: the one that failed because it doesn't have a dependency | 00:07 |
| *** jcooley_ has quit IRC | 00:08 | |
| jog0 | jeblair: thanks, too bad I didn't want the my patch to land | 00:09 |
| zaro | mordred: how to fix that? | 00:09 |
| *** jcooley_ has joined #openstack-infra | 00:09 | |
| zaro | mordred: i'm restarting review-dev | 00:09 |
| *** thuc_ has quit IRC | 00:10 | |
| zaro | clarify, just restarting gerrit | 00:10 |
| mordred | zaro: puppet. my current ssh key in puppet is different than the one that was there pre-christmas - so I can't see in | 00:11 |
| *** ken1ohmichi has joined #openstack-infra | 00:11 | |
| mordred | clarkb: would you prefer me to do the manage-projects run? I'm happy to do that? | 00:11 |
| mordred | clarkb: and while I do, I can verify that the config change doesn't break things - or I can not do that | 00:12 |
| *** CaptTofu has joined #openstack-infra | 00:12 | |
| clarkb | mordred: please do run it :) | 00:12 |
| *** wenlock has quit IRC | 00:12 | |
| zaro | mordred: clarkb is gonna take care of you. | 00:13 |
| *** yamahata__ has quit IRC | 00:13 | |
| mordred | awesome. I got it | 00:14 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Use the openstack pypi mirror for tripleo images. https://review.openstack.org/69309 | 00:14 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Run gate-tripleo-deploy on devstack-gate changes. https://review.openstack.org/69273 | 00:14 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Run gate-triple-deploy on tripleo-ci changes. https://review.openstack.org/69272 | 00:14 |
| clarkb | mordred: is camelot the new key? | 00:14 |
| mordred | clarkb: yes | 00:14 |
| clarkb | mordred: you should beable to log in now | 00:14 |
| mordred | clarkb: thank you | 00:14 |
| clarkb | I removed your old key for completeness | 00:14 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add support for Fedora 20 to nodepool https://review.openstack.org/69510 | 00:15 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Set a ratelimit for tripleo-test-cloud. https://review.openstack.org/68510 | 00:16 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Set appropriate rate-limit for RAX clouds. https://review.openstack.org/68509 | 00:16 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Set HP cloud rate limits. https://review.openstack.org/68512 | 00:16 |
| lifeless | fungi: rebased ^ | 00:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Add requirements gate for solum project https://review.openstack.org/66340 | 00:16 |
| clarkb | pleia2: do you have the ability to test that locally? last time we tried adding fedora to nodepool it broke because fedora, paramiko, and sudo do not play nice together | 00:16 |
| *** markmcclain has quit IRC | 00:16 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add manuals-upstream-translation-update to api-site https://review.openstack.org/66519 | 00:16 |
| clarkb | pleia2: fedora requires a tty for sudo and paramiko as used by nodepool doesn't allocate a tty | 00:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix CLI args for welcome-message https://review.openstack.org/66623 | 00:16 |
| *** jcooley_ has quit IRC | 00:16 | |
| clarkb | pleia2: though that was fedora 19 iirc | 00:16 |
| openstackgerrit | Michael Still proposed a change to openstack-infra/zuul: Implement a simple mysql reporter. https://review.openstack.org/65885 | 00:16 |
| pleia2 | clarkb: the plan for now is for it to go to the tripleo cloud, which I'll test it on, and it's a WIP for now since I still have some puppet work to do | 00:17 |
| *** dcramer_ has joined #openstack-infra | 00:17 | |
| pleia2 | clarkb: thanks though, I'll make note of those issues :) | 00:17 |
| clarkb | pleia2: you will test it against tripleo cloud without that change merging? (just clarifying as it was a bit of a mess last time, also the puppet work should be mostly done) | 00:18 |
| pleia2 | clarkb: yeah, planning on keeping it in WIP until I test it | 00:19 |
| clarkb | ++ | 00:19 |
| pleia2 | working with some redhat folks on this, so I wanted something up on review as I go along | 00:19 |
| *** SumitNaiksatam has joined #openstack-infra | 00:19 | |
| *** thuc has joined #openstack-infra | 00:22 | |
| openstackgerrit | Joshua Harlow proposed a change to openstack-infra/config: Name adjustments for taskflow jobs https://review.openstack.org/69513 | 00:22 |
| clarkb | sdague: there is a bug for the ipv6 thing with some of fungi's initial debugging | 00:23 |
| harlowja | hi guys, could i get a quick review of 69513, causing commits to fail due to the name change | 00:23 |
| *** hogepodge has quit IRC | 00:23 | |
| harlowja | much appreciated :) | 00:24 |
| clarkb | harlowja: how did that change merge if the names are different? | 00:24 |
| clarkb | also you can make these changes in a backward compat manner if you add the new envs, merge the config change, remove the old envs | 00:25 |
| harlowja | how did it merge, well failed on a recent taskflow review, so i guess config changes don't cause a check on the taskflow repo before merging? | 00:26 |
| clarkb | but I am curious how the change that updated tox.ini managed to merge if the envs were different | 00:26 |
| harlowja | that merged recently, before those envs were inplace (which afaik is recent) | 00:26 |
| *** matsuhashi has joined #openstack-infra | 00:27 | |
| harlowja | merged @ 11am (pst) - https://review.openstack.org/#/c/65135/ | 00:27 |
| harlowja | so change before that to tox.ini didn't update that review, a mistake i guess, then review goes through in 65135, then next review hits wrong env names | 00:27 |
| *** morganfainberg is now known as morganfainberg|z | 00:27 | |
| clarkb | oh it was a change to config not to tox.ini that broke | 00:28 |
| harlowja | one or the other, changes are brand new, so thats part of it | 00:28 |
| harlowja | from here on out it shouldn't be possible to do said things (once the corret names are in place) | 00:28 |
| harlowja | *correct | 00:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Add info about browser incompatibility to etherpad.rst https://review.openstack.org/67262 | 00:30 |
| harlowja | so afaik clarkb i got the names right this time :) | 00:32 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Split the config out into two files https://review.openstack.org/69503 | 00:33 |
| openstackgerrit | Aaron Greengrass proposed a change to openstack-infra/config: Enable PATH for exec functions https://review.openstack.org/69427 | 00:34 |
| *** dims has quit IRC | 00:34 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Split the config out into two files https://review.openstack.org/69503 | 00:36 |
| *** dims has joined #openstack-infra | 00:36 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Split the config out into two files https://review.openstack.org/69503 | 00:39 |
| openstackgerrit | A change was merged to openstack-infra/config: Add Periodic Translation Updates to ops-guide https://review.openstack.org/67022 | 00:40 |
| openstackgerrit | A change was merged to openstack-infra/config: gerritbot: Add API doc git notifications on #openstack-doc https://review.openstack.org/67573 | 00:41 |
| mordred | jeblair, clarkb: ok. https://review.openstack.org/#/c/69503/ has been emperically tested on review-dev. I'm comfortable with it now | 00:43 |
| *** ICmonitor has joined #openstack-infra | 00:43 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add gates for API projects and operations-guide https://review.openstack.org/67394 | 00:43 |
| clarkb | mordred: it is starred | 00:43 |
| *** ICmonitor1 has quit IRC | 00:44 | |
| fungi | mordred: it worked with gerrit 2.8 on review-dev, huh? neat | 00:44 |
| mordred | fungi: yah. woot | 00:44 |
| *** habdi has quit IRC | 00:45 | |
| *** zz_ewindisch is now known as ewindisch | 00:45 | |
| zaro | mordred: heading out until later. do you think you can restore jeepyb to master on review-dev when you are done? | 00:45 |
| *** virmitio has quit IRC | 00:45 | |
| harlowja | lol, so another question, we recently merged the usage of 'openstack_citest' in taskflow for db testing | 00:47 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove non-voting documentation gate job https://review.openstack.org/67702 | 00:47 |
| harlowja | we of course clean the db, but also just found that the following happens | 00:47 |
| harlowja | OperationalError: (OperationalError) database "openstack_citest" is being accessed by other users | 00:47 |
| harlowja | 2014-01-27 18:54:21.776 | DETAIL: There are 1 other session(s) using the database. | 00:47 |
| harlowja | if say nova is using that db | 00:47 |
| harlowja | and taskflow is also | 00:47 |
| harlowja | and taskflow and nova try to clean that db by doing, 'DROP DATABASE IF EXISTS openstack_citest' {} | 00:47 |
| fungi | harlowja: unit tests or devstack/tempest? | 00:48 |
| harlowja | unit, taskflow is just a library, so doesn't go through tempest | 00:48 |
| *** jcooley_ has joined #openstack-infra | 00:48 | |
| harlowja | http://logs.openstack.org/11/68111/7/check/gate-taskflow-python26/54df282/console.html | 00:48 |
| harlowja | interesting errors there, lol | 00:48 |
| clarkb | harlowja: only one tox process is run at a time | 00:48 |
| fungi | harlowja: we only run one jenkins worker per slave, so there is no possibility for nova and taskflow tests to run on the same slave at the same time | 00:49 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add storyboard-dev node with krotscheck on it https://review.openstack.org/69516 | 00:49 |
| harlowja | odd | 00:49 |
| fungi | or what clarkb said more eloquently and in fewer words | 00:49 |
| *** ICmonitor1 has joined #openstack-infra | 00:50 | |
| clarkb | harlowja: fungi: I suppose there could be a unittest process for nova hanging around if jenkins doesn't clean it up nicely | 00:50 |
| fungi | that's not to say that a particular slave won't necessarily have a hung process left over from a previous job locking some resource... but we consider that a problem and we clean it up when we hear about it | 00:50 |
| harlowja | ya, i put a big process lock around the taskflow tests | 00:50 |
| clarkb | harlowja: does taskflow run testr and have multiple tests that touch the DB? | 00:50 |
| * fungi will just stop typing--doesn't help that i'm repeating everything clarkb says ;) | 00:50 | |
| harlowja | right, but i put in process level locking to stop cross-process collisions | 00:50 |
| *** ICmonitor has quit IRC | 00:51 | |
| clarkb | fungi: no its good, makes me not crazy :) | 00:51 |
| clarkb | harlowja: please don't do that. | 00:51 |
| clarkb | harlowja: I wish that process level locking had never happened and that it would diaf | 00:51 |
| *** mfer has quit IRC | 00:51 | |
| clarkb | I worry that the ship has sailed | 00:51 |
| fungi | oslo sort of set an unfortunate precedent with that | 00:52 |
| *** ryanpetrello has joined #openstack-infra | 00:52 | |
| harlowja | k, so how should then we avoid each test messing up each others db | 00:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Add build job for Japanese Install Guide https://review.openstack.org/67481 | 00:52 |
| clarkb | now I am partially to blame because I didn't update mysql access quickly, but that is because there isn't a relatively secure way of doing that and I wanted people that knowwhat they are doing to help out | 00:52 |
| harlowja | *especially in testr | 00:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove link to puppet dashboard https://review.openstack.org/68724 | 00:52 |
| clarkb | harlowja: each test uses a different db | 00:52 |
| harlowja | k, is that possible? | 00:52 |
| clarkb | harlowja: for postgres it is | 00:52 |
| harlowja | thought only given access to openstack_citest | 00:52 |
| harlowja | k, so then for mysql? | 00:52 |
| clarkb | harlowja: for mysql, not currently because it requires giving that user add and drop | 00:52 |
| clarkb | drop is problematic | 00:52 |
| harlowja | right, so then still need process lock for mysql path | 00:53 |
| clarkb | harlowja: no | 00:53 |
| harlowja | ? | 00:53 |
| clarkb | harlowja: we need to give the user mysql perms to do it sanely | 00:53 |
| harlowja | k, whats missing to make that happen? | 00:53 |
| sdague | clarkb: so, I had another thought on the real db thing recently | 00:53 |
| clarkb | harlowja: we have a horde of mysql experts around and I secretly hoped one of them would say do it this way or write the change | 00:53 |
| clarkb | harlowja: but I haven't managed to get that out of them yet :) | 00:53 |
| harlowja | hmmm | 00:53 |
| sdague | which is kick them out of unit tests entirely, make people run functional tests on a d-g node | 00:54 |
| harlowja | why not just blow-away all dbs every time a test comes through? | 00:54 |
| clarkb | harlowja: thats the plan | 00:54 |
| clarkb | harlowja: but the tests need to be able to do it themselves | 00:54 |
| clarkb | harlowja: because infra shouldn't care | 00:54 |
| clarkb | sdague: I would be on board with that | 00:54 |
| *** oubiwann_ has joined #openstack-infra | 00:54 | |
| *** jcooley_ has quit IRC | 00:54 | |
| clarkb | sdague: but, that adds another slave to the mix for every patchset | 00:54 |
| sdague | it would also massively simplify the tox install for devs | 00:54 |
| sdague | clarkb: yes | 00:55 |
| harlowja | hmmm, so in the meantime, should i remove these tests, nice to have them to make sure sqlalachemy is working fine? | 00:55 |
| clarkb | sdague: which I am trying to reverse the tide on :) | 00:55 |
| harlowja | *with real dbs (not sqllite) | 00:55 |
| jeblair | clarkb, sdague: but does that really address the fundamental problem? you'd still want to paralellize those db tests running under testr | 00:55 |
| jeblair | clarkb, sdague: regardless of what kind of node they are running on | 00:55 |
| clarkb | sdague: after the gate thrash last week I think it is insane that we need so many slaves per change | 00:55 |
| clarkb | jeblair: it addresses the mysql access problem | 00:55 |
| jeblair | clarkb: how? | 00:55 |
| clarkb | jeblair: unlike postgres mysql has no good way of saying give this account access to add and drop new DBs | 00:56 |
| mordred | clarkb: not true | 00:56 |
| clarkb | mysql can only say add and drop anything globally | 00:56 |
| clarkb | mordred: you keep saying that but no one will tell me how to fix this problem | 00:56 |
| jeblair | clarkb: why is that a problem? | 00:56 |
| clarkb | mordred: and all of the docs are ^ | 00:56 |
| mordred | it's fixed | 00:56 |
| mordred | it's not a problem | 00:56 |
| clarkb | jeblair: if we give openstackci_test DROP in mysql it can drop anything | 00:56 |
| mordred | the key is, you can grant those permissions on the db once it's been created | 00:56 |
| clarkb | mordred: right thats the problem | 00:56 |
| clarkb | mordred: we want the tests to create arbitrary DBs themselves | 00:57 |
| mordred | create db, add permission, drop db | 00:57 |
| mordred | oh ARBITRARY dbs | 00:57 |
| harlowja | arbitary dbs FTW | 00:57 |
| harlowja | *to avoid name collisions | 00:57 |
| clarkb | postgres does this just fine as you can say DB creator gets drop and openstackci_test can add | 00:57 |
| clarkb | mysql doesn't know how to do that | 00:57 |
| mordred | yeah. with you | 00:57 |
| jeblair | clarkb: there's no db we care about other than 'mysql' itself | 00:57 |
| clarkb | jeblair: the information schema | 00:57 |
| jeblair | clarkb: and all nova unit test slaves are single-use. | 00:58 |
| clarkb | and mysql itself | 00:58 |
| mordred | you can't drop i_s | 00:58 |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Add jenkins master name to the logstash records https://review.openstack.org/67508 | 00:58 |
| jeblair | clarkb: so i actually don't care if the tests drop mysql itself. | 00:58 |
| clarkb | mordred: I thought hub_cap managed to do it | 00:58 |
| clarkb | jeblair: good point | 00:58 |
| mordred | clarkb: nah. he managed to do the thing I was talking about | 00:58 |
| clarkb | jeblair: so we just accept that tsets can be silly and throw them away | 00:58 |
| mordred | we already accept that I think | 00:58 |
| jeblair | clarkb: yep. | 00:58 |
| clarkb | mordred: not on the static hosts | 00:58 |
| harlowja | hmmm, bad hub_cap | 00:58 |
| clarkb | but those are going away | 00:59 |
| mordred | clarkb: on dynamic hosts | 00:59 |
| mordred | yeah | 00:59 |
| jeblair | the static hosts are dead to me | 00:59 |
| mordred | of course, I STILL think | 00:59 |
| mordred | that the unittests shoudl use drizzle and start/stop a new db server when they need isolation | 00:59 |
| jeblair | mordred: ++ | 00:59 |
| clarkb | :) | 00:59 |
| mordred | so that we don't need a globa db at all | 00:59 |
| *** ryanpetrello has quit IRC | 00:59 | |
| mordred | I mean, for the love of god, we spent the first year of the project fixing all the crap in mysql to allow for non-root installs to work sanely | 00:59 |
| harlowja | or have a script that can run per project to do test-env setup, teardown | 01:00 |
| *** habdi has joined #openstack-infra | 01:00 | |
| mordred | harlowja: that's called a fixture | 01:00 |
| harlowja | but i'd almost call setting up mysql something different | 01:00 |
| harlowja | or drizzle, or whatever | 01:00 |
| mordred | and you want it per unit test - and all we need is a fixture that does "get me a new port; drizzled -p $new_port; add_cleanup(kill drizzled)" | 01:00 |
| sdague | so the reason I'd like to change this is the fact that in order to pep8 nova, you have to install postgresql | 01:01 |
| mordred | that's my pint - it's only different because mysql and postgres both assume they are a heavyweight install | 01:01 |
| jeblair | mordred: did you just write it? | 01:01 |
| mordred | jeblair: pretty much | 01:01 |
| sdague | which is causing lots of people to not do that | 01:01 |
| clarkb | sdague: no, you have to install the client bindings its different | 01:01 |
| clarkb | very different | 01:01 |
| mordred | drizzle, on the other hand, does not need to be installed - it can run in local dirs | 01:01 |
| harlowja | ah, thats nice | 01:01 |
| mordred | clarkb: it's not if you run osx | 01:01 |
| sdague | clarkb: fair... however, I think it's time to rethink that | 01:01 |
| mordred | and dont' have a package manager so C libraries freak you out | 01:01 |
| sdague | because I think it makes sense to have a class of tests between the unit tests and tempest | 01:02 |
| mordred | sdague: I do too | 01:02 |
| harlowja | me too | 01:02 |
| mordred | swift has their functional tests, which I think make sense | 01:02 |
| sdague | right, exactly | 01:02 |
| mordred | glance and keystone each do $CRAZY to try to do that | 01:02 |
| harlowja | cause the unit tests can really run with sqlite, but its a nice middleground to run with postgres/mysql (real dbs) | 01:03 |
| fungi | another project has recently added functional tests similar to swift's too | 01:03 |
| clarkb | sdague: I don't disagree, but I think it is insane to spin up 12 nodes to test nova delete them all and do it all over again when pep8 fails | 01:03 |
| mordred | screw sqlite | 01:03 |
| sdague | and I think the bulk of the real db stuff that nova does actually should go into functional | 01:03 |
| *** mrodden has quit IRC | 01:03 | |
| mordred | :) | 01:03 |
| sdague | clarkb: sure | 01:03 |
| sdague | but that's about early pre-flight testing | 01:03 |
| jeblair | mordred: you can't run most of this stuff on osx so i don't think it makes sense to optomize the whole dev environment around it. i'll buy some of the other arguments, but not that one. | 01:03 |
| sdague | honestly, I don't think we should allocate any test resources before pep8 runs | 01:03 |
| clarkb | it is also about all of that extra testing we do that fails 100% fo the time... | 01:04 |
| sdague | jeblair: this isn't really the osx case, at least not in my thinking | 01:04 |
| mordred | jeblair: I agree - I was just responding to clarkb's comment about "it's just a library" | 01:04 |
| *** dcramer_ has quit IRC | 01:04 | |
| harlowja | clarkb should i leave the process locking code in, still trying to figure out if its ok to have that? disable those tests? | 01:04 |
| jeblair | sdague: that's fine. but mordred's statement required a response. :) | 01:04 |
| sdague | jeblair: sure :) | 01:04 |
| clarkb | harlowja: I am about to push a change to 'fix' mysql | 01:05 |
| harlowja | ah | 01:05 |
| harlowja | hmmm | 01:05 |
| harlowja | cool | 01:05 |
| clarkb | harlowja: if we can get reviewers for that then I think you can remove it otherwise keep it until mysql is happy | 01:05 |
| sdague | but we do get kind of crazy with what gets installed in tox venv | 01:05 |
| harlowja | kk | 01:05 |
| sdague | which means... oh let me pep8 this new tree.... oh, right, time to make a pot of coffee | 01:05 |
| sdague | and lunch | 01:05 |
| *** masayukig has quit IRC | 01:06 | |
| harlowja | sdague do u know if nova or others have different venvs for different types of plugins (one for mysql, one for postgres...), i believe they do this for tempest | 01:06 |
| sdague | harlowja: we don't have different venvs in tempest | 01:06 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Fix python 3 incompatibility in _get_import_type https://review.openstack.org/68849 | 01:06 |
| *** thuc has quit IRC | 01:06 | |
| harlowja | *sorry, not in that sense, a environment that uses postgres then | 01:06 |
| sdague | configuration is done in separate step | 01:06 |
| sdague | harlowja: yeh, look at a nova job | 01:07 |
| harlowja | right, so then the configuration step has knowledge of the variations in the nova code/requirements | 01:07 |
| sdague | you'll see the pg variant | 01:07 |
| sdague | harlowja: well... only by fiat | 01:08 |
| sdague | that's all done at a very different level | 01:08 |
| *** alexpilotti_ has quit IRC | 01:08 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Update ProjectGroups API to consume ID's rather than names. https://review.openstack.org/68540 | 01:09 |
| mordred | that pep8 requires mysql or postgres to be installed is a bug and should be fixed | 01:09 |
| harlowja | sdague ya, so many levels :-P | 01:09 |
| mordred | period. there is no reason that should be the case. whatever is causing it is doing something wrong | 01:09 |
| *** masayukig has joined #openstack-infra | 01:09 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Give test mysql user create and drop perms. https://review.openstack.org/69519 | 01:09 |
| clarkb | mordred: ^ how does that look? | 01:09 |
| sdague | mordred: deps = -r{toxinidir}/tools/pip-requires | 01:10 |
| sdague | -r{toxinidir}/tools/test-requires | 01:10 |
| sdague | that's what's doing it | 01:10 |
| clarkb | mordred: sdague hacking causes it | 01:10 |
| clarkb | because hacking imports stuff | 01:10 |
| harlowja | clarkb while u are at it, https://review.openstack.org/#/c/69513/ would be superb if u don't mind :) | 01:10 |
| mordred | no - he's talking about the tox env | 01:10 |
| clarkb | mordred: yes, the tox env needs it because hacking | 01:10 |
| mordred | hacking importing stuff can be worked around | 01:10 |
| sdague | clarkb: it needs it for pep8 | 01:10 |
| *** yamahata has joined #openstack-infra | 01:10 | |
| clarkb | if hacking didn't do import checks you can remove test-requires from the env | 01:10 |
| clarkb | and explicitly state the pep8/hacking/flake8 deps | 01:11 |
| clarkb | sdague: pep8 doesn't import stuff iirc | 01:11 |
| sdague | clarkb: hmmm... maybe? | 01:11 |
| jeblair | clarkb: yes, this is my understanding. | 01:11 |
| mordred | right. it does that to support "only import modules" | 01:11 |
| sdague | oh, right, that is actually the osx problem | 01:11 |
| mordred | flake8 runs fine on nova with no libs - I just tested it | 01:12 |
| fungi | does flake8 import stuff to do pyflakes-like analysis? | 01:12 |
| sdague | because there is a way to check that without eval in linux | 01:12 |
| fungi | ahh, retract my question | 01:12 |
| mordred | sdague: how about add in an attempt to do the linux version | 01:12 |
| mordred | and if the linux version doesn't work, skip the modules test | 01:12 |
| clarkb | mordred: nova is a bad test because site-packages are enabled | 01:12 |
| clarkb | which means you probably have libs somehwere | 01:12 |
| mordred | clarkb: I do not have ANYTHING installed on this machine | 01:12 |
| sdague | mordred: yeh, that's an interesting idea | 01:12 |
| fungi | apt-get remove --purge coreutils | 01:12 |
| mordred | I just re-installed it :) | 01:12 |
| fungi | ;) | 01:13 |
| *** CaptTofu has quit IRC | 01:13 | |
| sdague | I'd have to figure out where that fix is | 01:13 |
| mordred | sdague: and if that means taht osx people have a slightly higher annoyance rate with the gate catching pep8 things that local didn't catch, scrwe it | 01:13 |
| sdague | if we did that, we'd definitely want early abort on pep8 in check | 01:13 |
| *** CaptTofu has joined #openstack-infra | 01:13 | |
| sdague | because we'd get a bunch more bad pep8 | 01:13 |
| sdague | because way too many people on this project are mac people :) | 01:13 |
| fungi | people running non-free operating systems can be burdened to run a free operating system in a virtual machine for testing code meant to run on a free operating system, right? | 01:14 |
| jeblair | sdague: i agree that too many people on this project are mac people. | 01:14 |
| mordred | we need to be able to express grouping and dependency differently in zuul to do early-fail-on-pep8 | 01:14 |
| jeblair | mordred: no we don't | 01:14 |
| clarkb | jeblair: mordred: not since flake8 | 01:14 |
| mordred | with templates? | 01:14 |
| clarkb | since it bundles all of the cheap static checks together | 01:14 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Simple round trip API integration with storyboard-api https://review.openstack.org/68528 | 01:14 |
| jeblair | mordred: we only need to do the thing i suggested the last time you brought this up: allow jobs to be marked as able to cause early aborts. | 01:14 |
| jeblair | mordred: at least, that's if you want to do 'abort' on failure, rather than wait-for-success | 01:15 |
| harlowja | or i think jog0 was thinking that maybe someday in the future we can have a smart reformatter that fixes all hacking errors automagically, thats another solution to the pep8 problem :-P | 01:15 |
| sdague | jeblair: so that would still allocated the d-g nodes | 01:15 |
| harlowja | reformat all the code automatically | 01:15 |
| jeblair | mordred: if you want to do wait-for-success, then yeah, you need another way of grouping | 01:15 |
| jog0 | harlowja: I see that as a locally run tool only | 01:15 |
| mordred | I think we were looking for wait-for-success - wanting to avoid allocating d-g nodes until pep8 worked | 01:15 |
| jeblair | sdague: is this really a very pressing problem though? | 01:15 |
| harlowja | jog0 but u could imagine it being hooked into CI | 01:16 |
| sdague | jeblair: I don't know | 01:16 |
| jog0 | harlowja: and I can imagine that being confusing | 01:16 |
| harlowja | i believe google hooks in gofmt | 01:16 |
| sdague | jeblair: which part being pressing? | 01:16 |
| jeblair | mordred: i would be happy to support that in zuul; i don't think it's trivial to implement; but i'm not sure i would want that for openstack. | 01:16 |
| *** _ruhe is now known as ruhe | 01:16 | |
| jog0 | IIRC thats usually a precommit hook, anway that is future vs now | 01:16 |
| sdague | I do think we should probably try to make the tox dev side of the equation simpler | 01:16 |
| fungi | harlowja: eventually zuul will just reimplement all our python mess in lisp on the fly and reassign commit authorship to itself, because we are fallible meatsacks | 01:17 |
| harlowja | woot | 01:17 |
| harlowja | +2 | 01:17 |
| harlowja | skynet ftw | 01:17 |
| mordred | meatsacks ftw | 01:17 |
| jeblair | sdague: are we so short of check nodes that we really want to wait for a pep8 run to finish before starting anything else? it will slow check results down, and sometimes we're waiting on nodes to run pep8 checks while we have devstack nodes. | 01:17 |
| *** CaptTofu has quit IRC | 01:18 | |
| clarkb | gah I started reviewing lifeless' nodepool stack in the wrong direction then found a -1 whcih needs addressing | 01:18 |
| clarkb | that'll teach me | 01:18 |
| sdague | jeblair: honestly, I think that's your call. But clarkb expressed some concern on the number of slaves back at the beginning of this thread | 01:18 |
| fungi | wow. somebody rechecked that ginormous series of nova patches that run for 3 hours and fail out on large-ops | 01:18 |
| clarkb | jeblair: sdague: I think one of the major reasons gate thrash was so horrible was node starvation | 01:18 |
| sdague | fungi: it's not a recheck | 01:18 |
| sdague | it's a dansmith patch series | 01:19 |
| fungi | sdague: oh, rebased? | 01:19 |
| clarkb | jeblair: sdague: I think we avoid that problem entirely if we didn't consume so many nodes and throw them away | 01:19 |
| *** VijayT has quit IRC | 01:19 | |
| sdague | yeh | 01:19 |
| fungi | heh | 01:19 |
| *** VijayT has joined #openstack-infra | 01:19 | |
| jog0 | where do pep8 jobs get run, I'm surprised we sometimes wait for pep8 jobs to run | 01:19 |
| jeblair | clarkb: we just made MAJOR changes to how we use nodes, and we're making even more. how about we try them out for a few minutes before we re-engineer again? | 01:19 |
| sdague | I am going to nudge him tomorrow to not push the whole thing again until patch #2 is passing largeops | 01:19 |
| sdague | jeblair: yeh, agreed, I think we can sort that one later | 01:20 |
| clarkb | jeblair: I am not suggesting a reengineer, you guys are :P | 01:20 |
| lifeless | clarkb: oh, there's a -1, doh. | 01:20 |
| sdague | jeblair: so.... about to get called to dinner. Hows the "need valid check" logic? | 01:20 |
| clarkb | jog0: on single use slaves for some projects | 01:20 |
| jeblair | clarkb: clearly we're not having the same conversation then. i'm going to stop participating in it. :) goodnight! | 01:20 |
| dansmith | sdague: nudge me? | 01:20 |
| clarkb | jog0: on all projects | 01:20 |
| jeblair | sdague: did you see my a/b/c thing above? | 01:21 |
| clarkb | jog0: er soon on all projects | 01:21 |
| sdague | apparenlyt not | 01:21 |
| jog0 | clarkb: we run one pep8 on a node at a time | 01:21 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Changed default API base to match WSGI module https://review.openstack.org/69525 | 01:21 |
| clarkb | jeblair: I was responding to the idea of adding a completely new set of tests to the gate | 01:21 |
| clarkb | I agree we need the tests but should be careful about simply throwing nodes at the problem | 01:21 |
| sdague | yeh, that was a chunk when I ran away for a bit | 01:21 |
| jeblair | 23:53 < jeblair> (a) clarkb is looking into fixing gerrit so the timestamp is updated | 01:21 |
| jeblair | 23:54 < jeblair> (b) we can have zuul clear the check vote (pro: easy; con: adds an e23:54 < jeblair> (c) we search for "Works for me" and "Doesn't seem to work" comments from jenkins | 01:21 |
| jeblair | 23:55 < jeblair> (a) is a change to gerrit, (c) is a change to zuul, (b) is a zuul config change | 01:21 |
| jog0 | clarkb: interesting, thanks | 01:22 |
| jeblair | oops | 01:22 |
| mordred | clarkb, sdague we could build a special pep8 hivemind ... | 01:22 |
| jeblair | (b) adds an extra entry in the gerrit log for each change | 01:22 |
| clarkb | lifeless: I take it you are addressing that comment? | 01:23 |
| jeblair | sdague: clarkb is working on (a); it's uncertain and i would not expect resolution there for at least 2 weeks | 01:23 |
| lifeless | clarkb: I'll see if I can find someone to do so, and if not I will get to it when I get a chance to cycle back around | 01:23 |
| clarkb | lifeless: ok | 01:23 |
| jeblair | sdague: (c) we can probably do fairly quickly, but is ugly | 01:23 |
| lifeless | clarkb: but I have a tonne of urgent to do right now | 01:23 |
| jeblair | sdague: (b) is here: https://review.openstack.org/#/c/69508/ | 01:23 |
| lifeless | clarkb: so if you want to fix it, please go right ahead | 01:23 |
| clarkb | jeblair: fungi: for the nodepool stack before the -1, shoudl I go ahead and review that or is the value only in the full stack? | 01:24 |
| jeblair | sdague: i'm leaning toward (b) for now, (a) when ready, (c) if a fails. | 01:24 |
| *** dkliban has quit IRC | 01:24 | |
| *** wenlock has joined #openstack-infra | 01:24 | |
| clarkb | mordred: can you review https://review.openstack.org/#/c/69519/ ? | 01:24 |
| openstackgerrit | Joshua Harlow proposed a change to openstack-infra/config: Name adjustments for taskflow jobs https://review.openstack.org/69513 | 01:24 |
| jeblair | clarkb: i'm not certain; i'd have to look at all the changes, but i think significant chunks of the stack would be dangereous to merge not-as-a-unit | 01:25 |
| fungi | clarkb: which was the -1? | 01:25 |
| dansmith | jeblair: was sdague referring to my patch #2 above, in regards to not pushing until largeops is working? | 01:25 |
| sdague | ok, well dinner time, I left a quick comment | 01:25 |
| mordred | clarkb: on it | 01:25 |
| clarkb | fungi: https://review.openstack.org/#/c/67985/2 | 01:25 |
| jeblair | sdague: i'm trying to ramp-up. we can change to 24 hours later | 01:26 |
| sdague | dansmith: so until we figure out why your patch series is blowing up on that test, on every piece, can we just push the first one that fails :) | 01:26 |
| sdague | jeblair: ok | 01:26 |
| jeblair | sdague: but i'd like to start with 48 just to exercise all the bits first | 01:26 |
| mordred | clarkb: wfm | 01:26 |
| sdague | jeblair: sure | 01:26 |
| sdague | then you can ignore my -1 | 01:26 |
| dansmith | sdague: I don't know which piece you're talking about | 01:26 |
| mordred | clarkb: btw - the jeepyb changes are good | 01:26 |
| sdague | but, dinner... later | 01:26 |
| jeblair | sdague: for instance, when we first rolled it out, it spammed several thousand comments, and i'm very glad it only did so on 5 changes. | 01:26 |
| dansmith | sdague: the first several pass | 01:26 |
| lifeless | clarkb: jeblair: replied | 01:27 |
| *** ekis_isa has joined #openstack-infra | 01:28 | |
| lifeless | the answer is 'we could, after the rest of the stack is merged' | 01:28 |
| lifeless | IIR the stack C | 01:28 |
| jeblair | lifeless: your response doesn't make sense to me :( | 01:29 |
| fungi | lifeless: just wondering why the 'if not block_on_delete' is being inserted inside the for loop rather than immediately before it | 01:30 |
| jeblair | fungi: exactly | 01:30 |
| fungi | it's two new lines either way | 01:30 |
| fungi | so the patch isn't any smaller | 01:30 |
| lifeless | jeblair: the patch *after it* splits the calling function in two | 01:30 |
| clarkb | I am just going to go through the stack up to the -1 and review without approvals | 01:31 |
| clarkb | we can sort out merging when ready to babysit | 01:31 |
| lifeless | jeblair: if we don't split the calling function in two, then we have to probe once, or everything will always get forced to periodic cleanup, which you wanted to avoid | 01:31 |
| lifeless | jeblair: once the calling function is split in two and periodic cleanup is tuned to expect this, we can indeed stop probing at that place altogether | 01:31 |
| jeblair | lifeless: i don't see how a single probe (which is going to fail to find the server) is going to make any difference | 01:32 |
| lifeless | jeblair: I am fine however you want this tweaked, you asked for my reasoning, there you have it | 01:32 |
| jeblair | lifeless: (and indeed, the fact that this is in place is one of those critical chunks of patches i was mentioning to clarkb -- it _would_ be bad to merge up to this patch but not beyond) | 01:33 |
| lifeless | jeblair: it may be faulty | 01:33 |
| lifeless | jeblair: but I tried rather hard to make the stack part-mergable | 01:33 |
| jeblair | lifeless: i like where it ends up; i'm not particularly interested in trying to run a version of nodepool with a partial stack; i think we should address that nit, then merge the whole thing and go from there. | 01:34 |
| lifeless | thats fine by me | 01:34 |
| lifeless | like I say, I can't do the changes right now - but I'm very happy for someone to take over | 01:34 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add storyboard-dev node with krotscheck and ruhe on it https://review.openstack.org/69516 | 01:34 |
| *** CaptTofu has joined #openstack-infra | 01:34 | |
| lifeless | [making the code move is trivial, checking it still works is what would take me more time] | 01:35 |
| *** jhesketh has joined #openstack-infra | 01:35 | |
| *** nosnos has joined #openstack-infra | 01:36 | |
| *** jcooley_ has joined #openstack-infra | 01:42 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Teach periodicCleanup how to do one provider. https://review.openstack.org/68299 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Move cron loading below provider loading. https://review.openstack.org/68300 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Move cron definition out of the inner loop. https://review.openstack.org/68301 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Decouple cron names from config file names. https://review.openstack.org/68302 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Run per-provider cleanup threads. https://review.openstack.org/68303 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Make cleanupServer optionally nonblocking. https://review.openstack.org/67985 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Split out the logic for deleting a nodedb node. https://review.openstack.org/68003 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Use the nonblocking cleanupServer. https://review.openstack.org/68004 | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Include check in fake.yaml. https://review.openstack.org/68295 | 01:45 |
| clarkb | how does that look | 01:45 |
| *** senk has joined #openstack-infra | 01:45 | |
| clarkb | hrm, I suppose return False should fall outside the unneeded else block | 01:45 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Teach periodicCleanup how to do one provider. https://review.openstack.org/68299 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Move cron loading below provider loading. https://review.openstack.org/68300 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Move cron definition out of the inner loop. https://review.openstack.org/68301 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Decouple cron names from config file names. https://review.openstack.org/68302 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Run per-provider cleanup threads. https://review.openstack.org/68303 | 01:46 |
| clarkb | sorry for the spam | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Make cleanupServer optionally nonblocking. https://review.openstack.org/67985 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Split out the logic for deleting a nodedb node. https://review.openstack.org/68003 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Use the nonblocking cleanupServer. https://review.openstack.org/68004 | 01:46 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Include check in fake.yaml. https://review.openstack.org/68295 | 01:46 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Changed default API base to match WSGI module https://review.openstack.org/69525 | 01:46 |
| *** VijayT has quit IRC | 01:47 | |
| *** jcooley_ has quit IRC | 01:48 | |
| clarkb | anyways I don't have jeepyb configured in a way I can test that easily and it is getting close to dinner time | 01:49 |
| *** ryanpetrello has joined #openstack-infra | 01:49 | |
| ewindisch | now that we've settled down a bit... just want to get this on the radar: https://bugs.launchpad.net/openstack-ci/+bug/1273264 | 01:50 |
| ewindisch | (request for 3rd-partying testing service account) | 01:51 |
| *** julim has quit IRC | 01:56 | |
| *** gokrokve has joined #openstack-infra | 01:57 | |
| *** ryanpetrello has quit IRC | 01:59 | |
| *** ianw has quit IRC | 01:59 | |
| *** ianw has joined #openstack-infra | 02:00 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Changed default API base to match WSGI module https://review.openstack.org/69525 | 02:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Record build_branch in logstash https://review.openstack.org/67498 | 02:01 |
| *** ruhe is now known as _ruhe | 02:03 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Remove configure_tempest.sh https://review.openstack.org/66922 | 02:03 |
| *** dcramer_ has joined #openstack-infra | 02:06 | |
| *** gyee has quit IRC | 02:07 | |
| *** gokrokve has quit IRC | 02:08 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Updated grunt proxy server to rewrite to WSGI in dev https://review.openstack.org/69529 | 02:08 |
| *** yaguang has joined #openstack-infra | 02:12 | |
| *** vkozhukalov has joined #openstack-infra | 02:15 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable ironicclient py33 tests voting https://review.openstack.org/68092 | 02:16 |
| *** sarob has joined #openstack-infra | 02:16 | |
| harlowja | thx clarkb | 02:17 |
| *** SnowDust has joined #openstack-infra | 02:17 | |
| harlowja | (for the review) | 02:17 |
| harlowja | should unstick taskflow reviews | 02:17 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add support for Test Stability with Junit https://review.openstack.org/68152 | 02:18 |
| *** ryanpetrello has joined #openstack-infra | 02:22 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Fix references to examples in api documentation https://review.openstack.org/67712 | 02:23 |
| *** dkliban has joined #openstack-infra | 02:24 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for test_server_rescue race bug 1226412 https://review.openstack.org/69530 | 02:24 |
| *** chandankumar_ has joined #openstack-infra | 02:29 | |
| *** chuck__ has joined #openstack-infra | 02:29 | |
| *** chuck__ has quit IRC | 02:29 | |
| *** chandankumar_ has quit IRC | 02:31 | |
| *** jcooley_ has joined #openstack-infra | 02:36 | |
| *** jcooley_ has quit IRC | 02:42 | |
| *** ewindisch is now known as zz_ewindisch | 02:43 | |
| SlickNik | fungi: around? | 02:45 |
| *** ryanpetrello has quit IRC | 02:46 | |
| fungi | SlickNik: sorta... what's up? | 02:47 |
| SlickNik | pypy tests are giving me a hard time for python-troveclient. Trying to figure out what's up with them. | 02:48 |
| harlowja | fungi would it be possible for u to approve https://review.openstack.org/#/c/69513/ so that i can retrigger this test to make sure the right tox names are matching | 02:48 |
| fungi | SlickNik: i saw that... testing manually earlier i got... | 02:48 |
| fungi | ImportError: unable to load extension module '/home/fungi/python-troveclient/testing/site-packages/lxml/etree.pypy-20.so': /home/fungi/python-troveclient/testing/site-packages/lxml/etree.pypy-20.so: undefined symbol: PyUnicode_Concat | 02:48 |
| fungi | Alex_Gaynor might have pypy-relevant suggestions there | 02:49 |
| clarkb | pypy isn't compatible with CPython CFFI, could be related to that | 02:50 |
| Alex_Gaynor | lxml is a c-extension, and it looks like the etree module isn't supported. there's a pure python etree compatible API in the stdlib though, so you might be able to fallback to that? | 02:51 |
| *** habdi has left #openstack-infra | 02:51 | |
| harlowja | thx fungi | 02:51 |
| Alex_Gaynor | if it were cffi it would work fine :-) | 02:51 |
| fungi | SlickNik: seems you came to the right place ;) | 02:51 |
| SlickNik | fungi / Alex_Gaynor: thanks... | 02:51 |
| *** rnirmal has quit IRC | 02:51 | |
| *** sarob has quit IRC | 02:51 | |
| SlickNik | I'll look into that. | 02:52 |
| *** sarob has joined #openstack-infra | 02:52 | |
| SlickNik | We're looking to pull out xml support in the short term, so I might just end up pulling the whole thing out (in case that proves easier). | 02:52 |
| *** sarob_ has joined #openstack-infra | 02:53 | |
| openstackgerrit | A change was merged to openstack-infra/config: Name adjustments for taskflow jobs https://review.openstack.org/69513 | 02:54 |
| *** rcleere has joined #openstack-infra | 02:55 | |
| harlowja | alright, now to see if it all goes fine :) | 02:56 |
| *** krotscheck has quit IRC | 02:56 | |
| *** sarob has quit IRC | 02:56 | |
| harlowja | fungi does it take a little while to push out that config? | 02:57 |
| harlowja | i think i remember it being hourly (or similar) | 02:57 |
| *** sarob_ has quit IRC | 02:58 | |
| SlickNik | fungi / clarkb: I think I'm going to make PyPy non-voting for troveclient, until we figure out what we're doing with xml. Once we have that done, I'll turn it on again. There're quite a few checkins blocked on that right now. What say? | 02:58 |
| *** CaptTofu has quit IRC | 03:00 | |
| *** wenlock_ has joined #openstack-infra | 03:00 | |
| fungi | harlowja: yeah, let's call it 45 minutes (puppet master updates its git clone every 15 minutes, jenkins masters update their puppet configuration every 10 minutes, jenkins-job-builder takes up to 10 minutes to run through all the current configuration) | 03:00 |
| harlowja | fungi thx, guess i'll head to gym then come back to retry this :) | 03:00 |
| *** dstanek has quit IRC | 03:01 | |
| fungi | SlickNik: sounds reasonable | 03:01 |
| *** gokrokve has joined #openstack-infra | 03:01 | |
| *** nati_ueno has quit IRC | 03:02 | |
| *** dkliban has quit IRC | 03:02 | |
| * fungi is gonna disappear for a while | 03:03 | |
| SlickNik | Thanks! later. | 03:03 |
| *** dkliban has joined #openstack-infra | 03:05 | |
| *** morganfainberg|z is now known as morganfainberg | 03:11 | |
| *** matsuhashi has quit IRC | 03:13 | |
| openstackgerrit | Eli Klein proposed a change to openstack-infra/jenkins-job-builder: Added clone-workspace publisher https://review.openstack.org/69532 | 03:14 |
| *** matsuhashi has joined #openstack-infra | 03:14 | |
| openstackgerrit | SlickNik proposed a change to openstack-infra/config: Make PyPy gate job non-voting for python-troveclient https://review.openstack.org/69533 | 03:16 |
| *** zz_ewindisch is now known as ewindisch | 03:16 | |
| *** emagana has quit IRC | 03:17 | |
| *** ewindisch is now known as zz_ewindisch | 03:17 | |
| *** emagana has joined #openstack-infra | 03:18 | |
| *** rwsu has quit IRC | 03:18 | |
| *** VijayT has joined #openstack-infra | 03:18 | |
| lifeless | are tripleo test nodes still allowed reuse ? | 03:19 |
| *** mrodden has joined #openstack-infra | 03:20 | |
| clarkb | nodepool doesnt reuse aiui. unless hmm iy may be a layout thing where you need dsvm in the name | 03:20 |
| clarkb | for the function | 03:20 |
| lifeless | it was reusing yesterday | 03:21 |
| lifeless | which I quite liked ;) | 03:21 |
| openstackgerrit | Eli Klein proposed a change to openstack-infra/jenkins-job-builder: Added clone-workspace publisher https://review.openstack.org/69532 | 03:22 |
| *** sarob has joined #openstack-infra | 03:22 | |
| *** SnowDust has quit IRC | 03:24 | |
| clarkb | lifeless: there may be a problematic bug here. IIRC nodepool will always do its best to delete nodes after they have been used, but to make them go into offiline mode we need to make sure the job name matches the string for offlineing in zuul | 03:24 |
| clarkb | jeblair: ^ is this a problem for all of the other single use test slaves too? | 03:25 |
| *** emagana has quit IRC | 03:25 | |
| SlickNik | https://review.openstack.org/69533 when you guys get a chance. (Making the pypy gate job non-voting for python-troveclient) | 03:26 |
| clarkb | lifeless: its in the codepaths for nodepool that you touched :) you should know what it does. IIRC if state isn't READY or HOLD there is a timeout before it is deleted | 03:26 |
| SlickNik | Thanks! | 03:26 |
| clarkb | SlickNik: any idea how that snuck past the gate? | 03:26 |
| lifeless | the node complete function deletes always | 03:26 |
| clarkb | or are you trying to get something in? | 03:26 |
| clarkb | lifeless: ya | 03:26 |
| lifeless | clarkb: so it should be deleting, and my stuff hasn't landed yet :) | 03:26 |
| lifeless | clarkb: which means I don't know why it did some reuse yesterday | 03:27 |
| clarkb | lifeless: there is a delay right? | 03:27 |
| lifeless | https://jenkins02.openstack.org/computer/tripleo-precise-tripleo-test-cloud-1220561/ has no reuse | 03:27 |
| *** yamahata has quit IRC | 03:27 | |
| clarkb | lifeless: it may have reused within the delay? | 03:27 |
| *** masayukig has quit IRC | 03:27 | |
| lifeless | clarkb: no, it tries to delete directly | 03:27 |
| lifeless | clarkb: periodic cleanup had a delay | 03:27 |
| clarkb | ah | 03:28 |
| *** sarob has quit IRC | 03:28 | |
| lifeless | there might be a race condition with unregistering the node from jenkins | 03:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Make PyPy gate job non-voting for python-troveclient https://review.openstack.org/69533 | 03:29 |
| SlickNik | clarkb: Not sure, seems like a disconnect. It's never actually succeeded. | 03:32 |
| *** ICmonitor has joined #openstack-infra | 03:32 | |
| *** ICmonitor1 has quit IRC | 03:32 | |
| SlickNik | clarkb: thanks much! | 03:33 |
| clarkb | mordred: https://review.openstack.org/#/c/68439/1/modules/openstack_project/templates/nodepool/nodepool.yaml.erb I was seeing cpu limits hit by nodepool in iad today. any chance you didn't bump those numbers with the ram bump? | 03:33 |
| *** morganfainberg is now known as morganfainberg|z | 03:36 | |
| *** thuc has joined #openstack-infra | 03:40 | |
| *** emagana has joined #openstack-infra | 03:40 | |
| *** thuc_ has joined #openstack-infra | 03:41 | |
| mikal | clarkb: you still around? | 03:42 |
| mikal | I need a sample of /home/gerrit2/projects.yaml post template substitution to test some code | 03:43 |
| clarkb | mikal: yes, reviewing all the things | 03:43 |
| mikal | Is that something that someone can grab for me? | 03:43 |
| * clarkb looks at what is templated | 03:43 | |
| clarkb | mikal: so I will just tell you waht the values are for http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/review.projects.yaml.erb | 03:44 |
| mikal | Actually, all the substitutions are in the first doc, yes? | 03:44 |
| mikal | I only care about the second doc, so perhaps I am ok | 03:44 |
| clarkb | mikal: local git dir is /var/lib/git, gerrit-host is review.openstack.org, ssh_project_key is /home/gerrit2/review_site/etc/ssh_project_key_id_rsa | 03:45 |
| *** thuc has quit IRC | 03:45 | |
| mikal | clarkb: ta | 03:45 |
| *** kraman has quit IRC | 03:47 | |
| *** matsuhashi has quit IRC | 03:49 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 03:50 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Lump all defaults reading into one place https://review.openstack.org/69502 | 03:51 |
| lifeless | can changes be approved which failed check ? | 03:52 |
| wenlock | clarkb: we've been testing on our enviornment with 51425 a bit. Not on the slave side so much but more on gerrit, jenkins::master, pastebin, puppetmaster | 03:52 |
| clarkb | lifeless: yes | 03:52 |
| lifeless | technically, not policy- cool | 03:52 |
| clarkb | lifeless: but, make sure you intend they should pass | 03:52 |
| clarkb | ya | 03:52 |
| wenlock | hopefully it passes checks | 03:52 |
| clarkb | wenlock: cool | 03:52 |
| lifeless | clarkb: dprince is keen to see tripleo-gate doing check votes | 03:53 |
| clarkb | ah | 03:53 |
| clarkb | lifeless: so jeblair and sdague intend on enforcing check passes before queuing in the gate, that isn't happening today | 03:53 |
| clarkb | but will happen near future probably, just keep that in mind | 03:53 |
| *** pcrews has quit IRC | 03:53 | |
| *** dstanek has joined #openstack-infra | 03:54 | |
| lifeless | clarkb: this job isn't in the gate at all though | 03:54 |
| *** praneshp has quit IRC | 03:54 | |
| clarkb | lifeless: won't matter once jeblair and sdague get that in | 03:54 |
| lifeless | ok | 03:54 |
| clarkb | lifeless: https://review.openstack.org/#/c/69157/ is that purely syntax? | 03:54 |
| clarkb | I mean I don't care, but trying to understand why it matters | 03:54 |
| lifeless | its'a url | 03:54 |
| lifeless | (jokes) | 03:54 |
| clarkb | lifeless: the change at that url :P | 03:54 |
| lifeless | oh, just easier to read | 03:54 |
| lifeless | foo>bar == foo > bar in shell | 03:55 |
| *** mriedem has quit IRC | 03:55 | |
| *** CaptTofu has joined #openstack-infra | 03:55 | |
| lifeless | like it says in the commit message | 03:55 |
| lifeless | 'Just easier to read.' | 03:55 |
| clarkb | right, I never remember where whitespace matters and was sure it didn[t matter there | 03:55 |
| lifeless | look up and down a couple lines | 03:55 |
| *** emagana has quit IRC | 03:56 | |
| *** AaronGr is now known as AaronGr_Zzz | 03:57 | |
| *** jcooley_ has joined #openstack-infra | 03:57 | |
| mrodden | at what point should i not feel bad about doing a --reset-author on someone elses patchset | 03:58 |
| *** slong has quit IRC | 03:58 | |
| mrodden | because i am pretty sure i have added more lines of code in unit tests than this entire patch was before i started hacking on it | 03:58 |
| lifeless | mrodden: when if you were them and putting however much input of theirs is left in, you wouldn't bother adding co-authored-by: | 03:58 |
| lifeless | mrodden: I would just add cab | 03:59 |
| mrodden | add myself on coauthored? | 03:59 |
| mrodden | or add them on cab and reset | 03:59 |
| mrodden | was going to do the latter | 04:00 |
| *** CaptTofu has quit IRC | 04:00 | |
| *** harlowja is now known as harlowja_away | 04:01 | |
| *** jcooley_ has quit IRC | 04:02 | |
| lifeless | mrodden: do you really care? | 04:05 |
| lifeless | credit is shared w/c-a-b | 04:05 |
| mrodden | yeah | 04:05 |
| mrodden | i mean, i dont really care | 04:05 |
| mrodden | credit is shared | 04:05 |
| mrodden | people keep asking for updates to the patch in review and i'm the only one updating it now... | 04:06 |
| *** markwash has joined #openstack-infra | 04:06 | |
| *** harlowja_away is now known as harlowja | 04:06 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 04:06 |
| wenlock | forgot my puppet-lint :< | 04:06 |
| mrodden | also going from python to ruby is weird | 04:06 |
| *** kraman has joined #openstack-infra | 04:07 | |
| lifeless | oh don't do that :0 | 04:08 |
| mrodden | ruby? | 04:08 |
| lifeless | amen | 04:09 |
| mrodden | its dynamic and fun like python | 04:09 |
| mrodden | but it really suffers from what i term "the perl problem" | 04:09 |
| mrodden | meaning, there are a billion ways to do something and non of them really seem obvious | 04:10 |
| openstackgerrit | A change was merged to openstack-infra/config: Move tuskar-ui to horizon program. https://review.openstack.org/68264 | 04:12 |
| clarkb | and with that I am done reviewing for the night | 04:12 |
| * clarkb & | 04:12 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 04:13 |
| wenlock | pesky white spaces | 04:13 |
| *** gokrokve has quit IRC | 04:16 | |
| *** gokrokve_ has joined #openstack-infra | 04:17 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 04:20 |
| harlowja | hmmm, got closer on the tox.ini envs, is there a way to have a tox env use a specifc python version | 04:20 |
| harlowja | got 'ERROR: InterpreterNotFound: python2.6' | 04:20 |
| harlowja | maybe have to specify in a yaml file somewhere to use 2.6 for these envs | 04:21 |
| harlowja | clarkb if i specify 'node: centos6' would that make this happen (usage of 2.6) | 04:21 |
| *** gokrokve_ has quit IRC | 04:22 | |
| harlowja | *for a specific job | 04:22 |
| *** thuc has joined #openstack-infra | 04:22 | |
| clarkb | you want centis6 for py26 | 04:22 |
| harlowja | kk | 04:22 |
| *** sarob has joined #openstack-infra | 04:23 | |
| openstackgerrit | Joshua Harlow proposed a change to openstack-infra/config: Ensure taskflow py26 environments are using centos6 node https://review.openstack.org/69534 | 04:24 |
| *** AaronGr_Zzz is now known as AaronGr | 04:24 | |
| harlowja | think that should do it | 04:25 |
| *** thuc_ has quit IRC | 04:25 | |
| wenlock | cool, checks passed | 04:25 |
| *** thuc has quit IRC | 04:26 | |
| harlowja | clarkb fungi if u guys don't mind checking that out, hopefully last change ;) | 04:27 |
| *** sarob has quit IRC | 04:28 | |
| *** jcooley_ has joined #openstack-infra | 04:28 | |
| openstackgerrit | Michael Still proposed a change to openstack-infra/jeepyb: Allow configurable mappings to different LP projects https://review.openstack.org/62074 | 04:29 |
| *** jcooley_ has quit IRC | 04:30 | |
| *** wenlock has quit IRC | 04:31 | |
| *** masayukig has joined #openstack-infra | 04:31 | |
| *** david-lyle has joined #openstack-infra | 04:34 | |
| *** talluri has quit IRC | 04:41 | |
| *** UtahDave has quit IRC | 04:43 | |
| *** markwash has quit IRC | 04:44 | |
| *** matsuhashi has joined #openstack-infra | 04:45 | |
| *** thuc has joined #openstack-infra | 04:45 | |
| *** gokrokve has joined #openstack-infra | 04:47 | |
| *** praneshp has joined #openstack-infra | 04:48 | |
| *** gokrokve_ has joined #openstack-infra | 04:49 | |
| *** weshay has quit IRC | 04:49 | |
| *** gokrokve has quit IRC | 04:51 | |
| *** gokrokve_ has quit IRC | 04:53 | |
| *** markwash has joined #openstack-infra | 04:56 | |
| *** kraman has quit IRC | 05:00 | |
| *** AlexF_ has joined #openstack-infra | 05:01 | |
| *** gokrokve has joined #openstack-infra | 05:02 | |
| *** thuc has quit IRC | 05:03 | |
| *** thuc has joined #openstack-infra | 05:04 | |
| *** ekis_isa has quit IRC | 05:04 | |
| *** jp_at_hp has quit IRC | 05:05 | |
| *** jcooley_ has joined #openstack-infra | 05:06 | |
| *** jcooley_ has quit IRC | 05:07 | |
| *** emagana has joined #openstack-infra | 05:07 | |
| *** rushiagr has joined #openstack-infra | 05:08 | |
| *** rushiagr has left #openstack-infra | 05:08 | |
| *** thuc has quit IRC | 05:08 | |
| *** gokrokve has quit IRC | 05:09 | |
| *** emagana has quit IRC | 05:12 | |
| *** thuc has joined #openstack-infra | 05:16 | |
| *** markwash has quit IRC | 05:22 | |
| *** talluri has joined #openstack-infra | 05:23 | |
| *** sarob has joined #openstack-infra | 05:23 | |
| *** kraman has joined #openstack-infra | 05:25 | |
| *** AlexF_ has quit IRC | 05:25 | |
| openstackgerrit | Kei YAMAZAKI proposed a change to openstack-infra/jenkins-job-builder: Added support for Ci Skip Plugin https://review.openstack.org/69540 | 05:28 |
| *** sarob has quit IRC | 05:28 | |
| *** nicedice_ has quit IRC | 05:29 | |
| kashyap | Heya, /me is new to infra. Trying to explore how to have Fedora as part of Devstack Gate jobs. | 05:29 |
| * kashyap is cloning Nodepool to see if he can understand something | 05:30 | |
| clarkb | first step is getting it into nodepool. pleia2 is working on it too | 05:31 |
| clarkb | there are paramiko lack of tty sudo problems on fedora | 05:32 |
| *** AlexF_ has joined #openstack-infra | 05:32 | |
| *** AlexF_ has quit IRC | 05:35 | |
| kashyap | Hi clarkb, thanks for the detail. | 05:36 |
| kashyap | clarkb, Is there a bug report or some such for the problem you outlined? | 05:36 |
| clarkb | no bug iirc. | 05:37 |
| kashyap | Or some who knows the nature of the problem? | 05:37 |
| clarkb | the nature of the problem is sudo requires a tty on fedora. paramiko does not allocate a tty so the nodepool node preparation scripts fail | 05:38 |
| openstackgerrit | Kei YAMAZAKI proposed a change to openstack-infra/jenkins-job-builder: Added support for Ci Skip Plugin https://review.openstack.org/69540 | 05:38 |
| kashyap | clarkb, Thanks for that. New to all of Jenkins and CI infra. /me comes from libvirt/kvm/qemu land | 05:39 |
| *** Ryan_Lane has joined #openstack-infra | 05:40 | |
| kashyap | pleia2, Hi, when you're around - can you please note about your thoughts/or any issues you encountered (apart from what Clark outlined here) on getting Fedora into Gate? | 05:45 |
| kashyap | I just started this etherpad to capture things at one place, please add any issues - https://etherpad.openstack.org/p/fedora-on-gate | 05:46 |
| clarkb | kashyap: pleia2 has a WIP change in gerrit to add fedora hosts to nodepool | 05:47 |
| *** AlexF_ has joined #openstack-infra | 05:48 | |
| *** sarob has joined #openstack-infra | 05:48 | |
| kashyap | Nice. /me is available to test patches, feel free to bug me if I can be of any help there. | 05:49 |
| *** oubiwann_ has quit IRC | 05:51 | |
| *** AlexF_ has quit IRC | 05:51 | |
| *** sarob has quit IRC | 05:53 | |
| *** zehicle_at_dell has joined #openstack-infra | 05:53 | |
| *** CaptTofu has joined #openstack-infra | 05:56 | |
| *** CaptTofu has quit IRC | 06:00 | |
| openstackgerrit | Christopher Yeoh proposed a change to openstack-infra/elastic-recheck: Adds query for bug 1273556 https://review.openstack.org/69542 | 06:01 |
| *** gokrokve has joined #openstack-infra | 06:02 | |
| *** thuc has quit IRC | 06:05 | |
| *** thuc has joined #openstack-infra | 06:06 | |
| *** gokrokve has quit IRC | 06:07 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jeepyb: fix manage_projects with MySQL-Python 1.2.5 https://review.openstack.org/69543 | 06:10 |
| *** thuc has quit IRC | 06:10 | |
| *** harlowja is now known as harlowja_away | 06:10 | |
| *** jcoufal has joined #openstack-infra | 06:12 | |
| *** reed has quit IRC | 06:13 | |
| *** kraman has quit IRC | 06:14 | |
| *** AlexF_ has joined #openstack-infra | 06:20 | |
| *** mrda is now known as mrda_away | 06:34 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 06:43 | |
| *** kraman has joined #openstack-infra | 06:44 | |
| *** gsamfira has quit IRC | 06:46 | |
| *** rcleere has quit IRC | 06:47 | |
| *** gokrokve has joined #openstack-infra | 06:47 | |
| *** kraman has quit IRC | 06:49 | |
| *** emagana has joined #openstack-infra | 06:50 | |
| *** vkozhukalov has quit IRC | 06:51 | |
| *** gokrokve has quit IRC | 06:52 | |
| *** nati_ueno has joined #openstack-infra | 06:54 | |
| *** rcleere has joined #openstack-infra | 06:56 | |
| *** sarob has joined #openstack-infra | 06:58 | |
| *** VijayT has quit IRC | 06:59 | |
| *** sarob has quit IRC | 07:03 | |
| *** nati_ueno has quit IRC | 07:07 | |
| *** nati_ueno has joined #openstack-infra | 07:08 | |
| *** starmer has joined #openstack-infra | 07:11 | |
| *** yolanda_ has joined #openstack-infra | 07:13 | |
| openstackgerrit | A change was merged to openstack/requirements: Require httplib2>=0.7.5 https://review.openstack.org/66486 | 07:15 |
| *** coolsvap has joined #openstack-infra | 07:17 | |
| *** kruskakli has joined #openstack-infra | 07:17 | |
| *** afazekas has joined #openstack-infra | 07:17 | |
| *** starmer_ has joined #openstack-infra | 07:21 | |
| *** starmer has quit IRC | 07:24 | |
| *** katyafervent is now known as katyafervent_awa | 07:28 | |
| *** vkozhukalov has joined #openstack-infra | 07:33 | |
| *** senk has quit IRC | 07:35 | |
| openstackgerrit | Ivan Melnikov proposed a change to openstack-dev/hacking: More portable way to detect modules for H302 https://review.openstack.org/68858 | 07:37 |
| *** flaper87|afk is now known as flaper87 | 07:44 | |
| *** gokrokve has joined #openstack-infra | 07:47 | |
| *** AlexF_ has quit IRC | 07:50 | |
| *** rcleere has quit IRC | 07:51 | |
| *** gokrokve has quit IRC | 07:52 | |
| *** markmc has joined #openstack-infra | 07:56 | |
| *** CaptTofu has joined #openstack-infra | 07:57 | |
| *** CaptTofu has quit IRC | 08:01 | |
| *** nati_uen_ has joined #openstack-infra | 08:02 | |
| *** luqas has joined #openstack-infra | 08:04 | |
| *** nati_ueno has quit IRC | 08:05 | |
| *** praneshp has quit IRC | 08:12 | |
| *** yamahata has joined #openstack-infra | 08:13 | |
| *** sdake has quit IRC | 08:14 | |
| *** sdake has joined #openstack-infra | 08:15 | |
| *** nati_uen_ has quit IRC | 08:15 | |
| *** vjay has joined #openstack-infra | 08:15 | |
| *** hashar has joined #openstack-infra | 08:15 | |
| *** nati_ueno has joined #openstack-infra | 08:16 | |
| *** ICmonitor has quit IRC | 08:17 | |
| vjay | What is the process to be followed when jenkins is voting a review submitted as "-1" for test failures not related to the actual fix? | 08:18 |
| *** ICmonitor has joined #openstack-infra | 08:19 | |
| *** mrodden has quit IRC | 08:19 | |
| *** mrodden has joined #openstack-infra | 08:19 | |
| vjay | It is failing due to bugs 1249065 & 1254890. But "Elastic recheck" does not add an entry in the Gerrit. | 08:20 |
| *** lttrl has joined #openstack-infra | 08:23 | |
| *** yassine has joined #openstack-infra | 08:23 | |
| *** david-lyle has quit IRC | 08:27 | |
| *** e0ne has joined #openstack-infra | 08:27 | |
| *** lttrl has quit IRC | 08:33 | |
| *** dizquierdo has joined #openstack-infra | 08:39 | |
| *** e0ne has quit IRC | 08:44 | |
| *** e0ne has joined #openstack-infra | 08:45 | |
| *** pblaho has joined #openstack-infra | 08:47 | |
| *** starmer_ has quit IRC | 08:47 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: test dequeue on abandoned changes https://review.openstack.org/65476 | 08:49 |
| *** e0ne has quit IRC | 08:49 | |
| *** _ruhe is now known as ruhe | 08:51 | |
| *** vkozhukalov has quit IRC | 08:51 | |
| *** Ryan_Lane has quit IRC | 08:51 | |
| *** Ryan_Lane has joined #openstack-infra | 08:52 | |
| *** starmer has joined #openstack-infra | 08:52 | |
| *** Ryan_Lane has quit IRC | 08:52 | |
| *** DinaBelova_ is now known as DinaBelova | 08:53 | |
| *** starmer has quit IRC | 08:55 | |
| *** jpich has joined #openstack-infra | 08:59 | |
| *** vjay has quit IRC | 09:04 | |
| *** ken1ohmichi has quit IRC | 09:05 | |
| *** derekh has joined #openstack-infra | 09:05 | |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 09:08 | |
| *** nati_uen_ has joined #openstack-infra | 09:09 | |
| *** nati_uen_ has quit IRC | 09:09 | |
| *** matsuhashi has quit IRC | 09:09 | |
| *** nati_uen_ has joined #openstack-infra | 09:09 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: dequeue abandoned changes https://review.openstack.org/65467 | 09:10 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: dequeue abandoned changes https://review.openstack.org/65467 | 09:12 |
| *** nati_ueno has quit IRC | 09:12 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Enable precise-backports on tripleo test nodes https://review.openstack.org/67958 | 09:13 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Add some dependencies required by toci https://review.openstack.org/67685 | 09:13 |
| *** matsuhas_ has joined #openstack-infra | 09:14 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: test dequeue on abandoned changes https://review.openstack.org/65476 | 09:14 |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 09:15 | |
| *** rossella_s has joined #openstack-infra | 09:18 | |
| *** marun has quit IRC | 09:19 | |
| *** ruhe is now known as _ruhe | 09:20 | |
| *** e0ne has joined #openstack-infra | 09:28 | |
| *** wenlock_ has quit IRC | 09:30 | |
| *** amotoki has joined #openstack-infra | 09:33 | |
| *** johnthetubaguy has joined #openstack-infra | 09:34 | |
| *** jooools has joined #openstack-infra | 09:40 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Fix up `zuul --version` https://review.openstack.org/66629 | 09:44 |
| *** gokrokve has joined #openstack-infra | 09:47 | |
| *** nati_ueno has joined #openstack-infra | 09:51 | |
| *** gokrokve has quit IRC | 09:52 | |
| *** nati_uen_ has quit IRC | 09:53 | |
| *** luqas has quit IRC | 09:57 | |
| *** nati_ueno has quit IRC | 09:57 | |
| *** CaptTofu has joined #openstack-infra | 09:57 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 09:58 | |
| *** che-arne has joined #openstack-infra | 09:59 | |
| *** CaptTofu has quit IRC | 10:02 | |
| *** e0ne_ has joined #openstack-infra | 10:06 | |
| *** wenlock has joined #openstack-infra | 10:08 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: Zuul trigger no more send GERRIT_* env variable https://review.openstack.org/69566 | 10:09 |
| *** e0ne has quit IRC | 10:10 | |
| *** luqas has joined #openstack-infra | 10:19 | |
| *** jhesketh has quit IRC | 10:20 | |
| *** jhesketh__ has quit IRC | 10:20 | |
| *** matsuhas_ has quit IRC | 10:22 | |
| *** lcestari has joined #openstack-infra | 10:27 | |
| *** max_lobur_afk is now known as max_lobur | 10:29 | |
| *** jp_at_hp has joined #openstack-infra | 10:29 | |
| *** matsuhashi has joined #openstack-infra | 10:31 | |
| *** jhesketh has joined #openstack-infra | 10:32 | |
| *** jhesketh__ has joined #openstack-infra | 10:32 | |
| *** vjay has joined #openstack-infra | 10:33 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/release-tools: Fix requirements installation for pip>=1.5 https://review.openstack.org/69572 | 10:33 |
| *** wenlock has quit IRC | 10:34 | |
| *** vkozhukalov has joined #openstack-infra | 10:37 | |
| SergeyLukjanov | ttx, ping | 10:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/release-tools: Fix requirements installation for pip>=1.5 https://review.openstack.org/69572 | 10:39 |
| ttx | SergeyLukjanov: pong | 10:39 |
| SergeyLukjanov | ttx, hi | 10:39 |
| SergeyLukjanov | ttx, the reason why release-tools fails to install is lazr.auth | 10:40 |
| ttx | SergeyLukjanov: I figured that much but couldn't tell WHY that was failing | 10:40 |
| SergeyLukjanov | ttx, lazr.auth is hosted on launchpad | 10:41 |
| ttx | uh | 10:41 |
| SergeyLukjanov | ttx, only download links on pypi | 10:41 |
| SergeyLukjanov | https://pypi.python.org/simple/lazr.authentication/ | 10:41 |
| ttx | ah. | 10:42 |
| ttx | SergeyLukjanov: I wonder how that ever worked, then | 10:42 |
| ttx | SergeyLukjanov: did we recently add tests ? | 10:42 |
| SergeyLukjanov | ttx, nope | 10:42 |
| SergeyLukjanov | pip was updated to 1.5 | 10:42 |
| SergeyLukjanov | there were tons of problems with it :) | 10:43 |
| ttx | looks like you need --allow-unverified | 10:43 |
| ttx | bit scary | 10:43 |
| SergeyLukjanov | check the previous patch set | 10:43 |
| SergeyLukjanov | it adds --allow-unverified | 10:44 |
| SergeyLukjanov | but isn't working too | 10:44 |
| SergeyLukjanov | https://review.openstack.org/#/c/66364/8/pypi_mirror/cmd/run_mirror.py adds all needed packages to pypi.o.o mirror | 10:44 |
| ttx | beh | 10:44 |
| SergeyLukjanov | patch set #1 works ok on my laptop with pip 1.5.2 | 10:45 |
| StevenK | Hm, I wonder if the pip 1.5 upgrade is the cause of git-review being so unhappy, then. | 10:45 |
| *** andreaf has joined #openstack-infra | 10:46 | |
| *** coolsvap has quit IRC | 10:47 | |
| ttx | SergeyLukjanov: ok.. it's not as if we were actually testing anything with it | 10:47 |
| *** gokrokve has joined #openstack-infra | 10:47 | |
| ttx | SergeyLukjanov: so i wonder if we should not strip requirements.txt | 10:47 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/release-tools: Fix requirements installation for pip>=1.5 https://review.openstack.org/69572 | 10:48 |
| SergeyLukjanov | probably we could add lazr.auth to the pypi.o.o/infra | 10:48 |
| ttx | that would be the alternative | 10:49 |
| SergeyLukjanov | heh, it works ok with the patch set #3 | 10:50 |
| ttx | heh. | 10:50 |
| SergeyLukjanov | https://jenkins02.openstack.org/job/gate-release-tools-pep8/6/ | 10:51 |
| SergeyLukjanov | random combination of --allow-abcd | 10:51 |
| ttx | OK, let's merge this then | 10:51 |
| *** gokrokve has quit IRC | 10:52 | |
| SergeyLukjanov | ttx, checking it now on my laptop with pip 1.5.2 | 10:53 |
| *** ArxCruz has joined #openstack-infra | 10:53 | |
| openstackgerrit | A change was merged to openstack-infra/release-tools: Fix requirements installation for pip>=1.5 https://review.openstack.org/69572 | 10:54 |
| *** DinaBelova is now known as DinaBelova_ | 10:58 | |
| *** coolsvap has joined #openstack-infra | 11:00 | |
| *** matsuhashi has quit IRC | 11:00 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Update documentation https://review.openstack.org/69211 | 11:02 |
| *** yamahata has quit IRC | 11:03 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: API tests for rest https://review.openstack.org/67447 | 11:04 |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 11:04 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 11:06 | |
| openstackgerrit | A change was merged to openstack-infra/release-tools: Added ignore to additional egg-info files https://review.openstack.org/68471 | 11:07 |
| *** boris-42 has quit IRC | 11:11 | |
| *** matsuhashi has joined #openstack-infra | 11:14 | |
| *** matsuhashi has quit IRC | 11:17 | |
| *** DinaBelova_ is now known as DinaBelova | 11:18 | |
| *** gokrokve has joined #openstack-infra | 11:19 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_a | 11:19 | |
| *** SergeyLukjanov_a is now known as SergeyLukjanov | 11:19 | |
| SergeyLukjanov | ttx, it works ok on 1.5.2 pip too | 11:19 |
| ttx | SergeyLukjanov: ok, all merged | 11:19 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Database fixture added https://review.openstack.org/69384 | 11:20 |
| *** gokrokve_ has joined #openstack-infra | 11:20 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Load projects from yaml file https://review.openstack.org/66280 | 11:21 |
| SergeyLukjanov | ttx, great | 11:21 |
| sdague | ttx: can we get the logging standards on the project agenda for today? I'd like to socialize with PTLs and figure out which ones are interested | 11:22 |
| ttx | sdague: sure, adding | 11:22 |
| *** gokrokve has quit IRC | 11:23 | |
| *** gokrokve_ has quit IRC | 11:25 | |
| *** coolsvap has quit IRC | 11:30 | |
| openstackgerrit | Julien Danjou proposed a change to openstack-dev/pbr: package: read a specific Python version requirement file https://review.openstack.org/63236 | 11:30 |
| *** rfolco has joined #openstack-infra | 11:36 | |
| *** gokrokve has joined #openstack-infra | 11:47 | |
| *** hashar has quit IRC | 11:49 | |
| *** gokrokve has quit IRC | 11:52 | |
| *** lttrl has joined #openstack-infra | 11:53 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 11:53 | |
| *** masayukig has quit IRC | 11:57 | |
| *** CaptTofu has joined #openstack-infra | 11:58 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 12:02 | |
| *** yamahata has joined #openstack-infra | 12:03 | |
| *** CaptTofu has quit IRC | 12:04 | |
| *** vjay has quit IRC | 12:05 | |
| *** hashar has joined #openstack-infra | 12:05 | |
| *** vipul has quit IRC | 12:10 | |
| *** vipul has joined #openstack-infra | 12:12 | |
| openstackgerrit | Max Lobur proposed a change to openstack/requirements: Add futures library to global requirements https://review.openstack.org/66349 | 12:12 |
| *** beagles has quit IRC | 12:15 | |
| *** emagana has quit IRC | 12:19 | |
| *** b3nt_pin has joined #openstack-infra | 12:20 | |
| *** jhesketh__ has quit IRC | 12:22 | |
| *** changbl has quit IRC | 12:22 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Bump hacking version in requirements https://review.openstack.org/49486 | 12:24 |
| *** jasondotstar has joined #openstack-infra | 12:28 | |
| *** b3nt_pin is now known as beagles | 12:29 | |
| *** boris-42 has joined #openstack-infra | 12:31 | |
| *** paul-- has joined #openstack-infra | 12:32 | |
| *** lttrl has quit IRC | 12:33 | |
| *** YorikSar has joined #openstack-infra | 12:36 | |
| *** luqas has quit IRC | 12:36 | |
| russellb | infra is so l33t that it crashes vim: https://code.google.com/p/vim/issues/detail?id=196 | 12:39 |
| YorikSar | Hello. Can anyone help me with an issue with Jenkins node: I get "Cannot fork" on precisepy3k-4 node. | 12:39 |
| YorikSar | http://logs.openstack.org/86/49486/2/check/gate-git-review-python33/ba40f80/console.html | 12:40 |
| YorikSar | And it's about to get the same result again. | 12:40 |
| *** max_lobur is now known as max_lobur_afk | 12:41 | |
| YorikSar | Tests for 2.6 and 2.7 were run on centos6-5 and precise22 with no such issue | 12:41 |
| *** jhesketh has quit IRC | 12:43 | |
| openstackgerrit | Ruslan Kamaldinov proposed a change to openstack-infra/storyboard: Added documentation for REST API layer. https://review.openstack.org/69212 | 12:43 |
| *** paul-- has quit IRC | 12:43 | |
| *** dkranz has joined #openstack-infra | 12:45 | |
| *** CaptTofu has joined #openstack-infra | 12:46 | |
| *** yaguang has quit IRC | 12:47 | |
| *** smarcet has joined #openstack-infra | 12:47 | |
| *** gokrokve has joined #openstack-infra | 12:47 | |
| *** vjay has joined #openstack-infra | 12:49 | |
| *** hashar has quit IRC | 12:51 | |
| sdague | russellb: that's pretty wild | 12:51 |
| russellb | :) | 12:51 |
| *** amotoki has quit IRC | 12:51 | |
| *** _ruhe is now known as ruhe | 12:51 | |
| russellb | i started looking at the code and then .... yeah | 12:51 |
| russellb | it wasn't obvious | 12:51 |
| *** gokrokve has quit IRC | 12:52 | |
| sdague | yeh, crazy. I wonder if their syntax highlighting is guessing the file wrong | 12:55 |
| sdague | yeh, I think it's the yaml parser, if you rename the file to nodepool.erb, it works | 12:57 |
| *** dprince has joined #openstack-infra | 12:59 | |
| *** markwash has joined #openstack-infra | 13:00 | |
| *** markmcclain has joined #openstack-infra | 13:00 | |
| *** heyongli has joined #openstack-infra | 13:00 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Database fixture added https://review.openstack.org/69384 | 13:02 |
| *** afazekas has quit IRC | 13:02 | |
| *** max_lobur_afk is now known as max_lobur | 13:04 | |
| *** nosnos has quit IRC | 13:08 | |
| russellb | sdague: cool, added that note to the bug | 13:09 |
| russellb | sdague: too bad the bug list looks like a ghost town, heh | 13:09 |
| russellb | i shouldn't talk ... | 13:09 |
| *** luqas has joined #openstack-infra | 13:10 | |
| sdague | heh | 13:11 |
| *** pblaho has quit IRC | 13:11 | |
| *** gokrokve has joined #openstack-infra | 13:12 | |
| *** gokrokve has quit IRC | 13:14 | |
| YorikSar | russellb: Have you tried more recent version of Vim? | 13:14 |
| YorikSar | russellb: I've just tried it on 7.4.131 and it works fine. | 13:15 |
| russellb | no, i haven't tried any other version | 13:15 |
| russellb | YorikSar: that's good to know though | 13:15 |
| *** afazekas has joined #openstack-infra | 13:15 | |
| *** dhellmann_ is now known as dhellmann | 13:16 | |
| *** chandankumar has joined #openstack-infra | 13:16 | |
| *** sandywalsh has joined #openstack-infra | 13:18 | |
| *** freyes has joined #openstack-infra | 13:18 | |
| *** luqas has quit IRC | 13:18 | |
| *** emagana has joined #openstack-infra | 13:19 | |
| *** markwash has quit IRC | 13:23 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Add a new node type for precise with saucy kernel https://review.openstack.org/69445 | 13:23 |
| *** pblaho has joined #openstack-infra | 13:24 | |
| *** katyafervent_awa is now known as katyafervent | 13:27 | |
| openstackgerrit | Julien Danjou proposed a change to openstack-dev/hacking: Allow to have noqa on H202 https://review.openstack.org/69596 | 13:31 |
| *** afazekas is now known as afazekas|mtg | 13:31 | |
| *** eharney has joined #openstack-infra | 13:33 | |
| *** beagles has quit IRC | 13:33 | |
| *** zhiyan has joined #openstack-infra | 13:34 | |
| *** NithyaG has joined #openstack-infra | 13:35 | |
| *** chandankumar has quit IRC | 13:36 | |
| salv-orlando | Hi, I have a question. I'm still chasing this kernel bug, or at least trying to understand the exact root cause (I'm talking about bug 1273386) | 13:36 |
| *** dhellmann is now known as dhellmann_ | 13:36 | |
| *** dhellmann_ is now known as dhellmann | 13:37 | |
| salv-orlando | Basically, kernel 3.2.0-58 was published on 2014/01/02 - How long would it take from there to be installed on all the instances we use for testing on the gate? idk how frequently they are recycled | 13:37 |
| salv-orlando | I'm just trying to understand whether the kernel crashes started with this new build or whether the root cause is something else | 13:38 |
| *** b3nt_pin has joined #openstack-infra | 13:39 | |
| *** miqui has joined #openstack-infra | 13:40 | |
| fungi | salv-orlando: uname -r on a random slave says "3.2.0-58-virtual #88-Ubuntu" | 13:41 |
| sdague | salv-orlando: so we rebuild the nodes nightly | 13:41 |
| sdague | fungi: I assume we'd have taken that update the day after it published? | 13:41 |
| fungi | salv-orlando: we sould have been using that for devstack runs within 24 hours of ubuntu releasing it into their repository | 13:41 |
| fungi | sdague: yes | 13:41 |
| sdague | or is that coming from the base image? | 13:41 |
| fungi | oh, hmmm | 13:42 |
| salv-orlando | so if the kernel was published on 2014/01/02 by 2014/01/03 it would have been on all our images | 13:42 |
| salv-orlando | The oldest log we have now is from Jan 15th | 13:42 |
| fungi | righth, ubuntu does funny things with making kernel packages require dist-upgrade. i'd have to check the base images to see what they're running, but i think we do upgrade the kernel when building the image (the image log would tell us too) | 13:42 |
| fungi | salv-orlando: i believe so, yes | 13:43 |
| fungi | i have to run out to an unanticipated appointment for a few hours though, so i can't really check much deeper right this moment | 13:43 |
| *** dkliban has quit IRC | 13:44 | |
| sdague | salv-orlando: so that's the oldest ES log | 13:44 |
| sdague | but we have raw logs before then | 13:45 |
| salv-orlando | sdague: yup, I managed to find them | 13:45 |
| salv-orlando | I'm now reading a log from jan 10th | 13:45 |
| salv-orlando | shame I can't find the kernel version without a kernel crash dump! do we log it somewhere else? | 13:45 |
| *** prad has joined #openstack-infra | 13:45 | |
| sdague | salv-orlando: probably not | 13:45 |
| salv-orlando | sdague: it seems we do just uname -m in devstack, it won't print kernel version | 13:46 |
| sdague | salv-orlando: yeh | 13:46 |
| sdague | there is also a pending change to dump the entire package list, which would help | 13:46 |
| *** b3nt_pin has quit IRC | 13:48 | |
| *** markwash has joined #openstack-infra | 13:49 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add Savanna screen file to logstash indexing https://review.openstack.org/69602 | 13:50 |
| *** dims has quit IRC | 13:50 | |
| *** emagana has quit IRC | 13:51 | |
| *** dims has joined #openstack-infra | 13:52 | |
| *** thuc has joined #openstack-infra | 13:53 | |
| *** thuc_ has joined #openstack-infra | 13:54 | |
| *** oubiwann_ has joined #openstack-infra | 13:54 | |
| *** b3nt_pin has joined #openstack-infra | 13:55 | |
| *** julim has joined #openstack-infra | 13:56 | |
| *** markmcclain has quit IRC | 13:57 | |
| *** markmcclain has joined #openstack-infra | 13:57 | |
| *** thuc has quit IRC | 13:58 | |
| openstackgerrit | A change was merged to openstack/requirements: Added a requirement to COIN PULP LP Modeler module https://review.openstack.org/53446 | 13:58 |
| *** oubiwann_ has quit IRC | 13:59 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Make cells job gating https://review.openstack.org/63537 | 13:59 |
| *** oubiwann_ has joined #openstack-infra | 14:01 | |
| *** dims has quit IRC | 14:03 | |
| *** chandankumar has joined #openstack-infra | 14:03 | |
| *** heyongli has quit IRC | 14:03 | |
| *** mestery has quit IRC | 14:03 | |
| *** dims has joined #openstack-infra | 14:04 | |
| *** gokrokve has joined #openstack-infra | 14:05 | |
| *** yaguang has joined #openstack-infra | 14:06 | |
| *** gokrokve_ has joined #openstack-infra | 14:08 | |
| *** dkranz has quit IRC | 14:09 | |
| *** chandankumar has quit IRC | 14:09 | |
| *** dcramer_ has quit IRC | 14:10 | |
| *** gokrokve has quit IRC | 14:11 | |
| openstackgerrit | godp1301 proposed a change to openstack-infra/jenkins-job-builder: Add best and worst status details for conditional-step plugin. https://review.openstack.org/69608 | 14:11 |
| *** afazekas|mtg has quit IRC | 14:11 | |
| *** prad has quit IRC | 14:12 | |
| *** dkehn_ has joined #openstack-infra | 14:14 | |
| *** pblaho has quit IRC | 14:14 | |
| *** mestery has joined #openstack-infra | 14:15 | |
| *** emagana has joined #openstack-infra | 14:17 | |
| *** afazekas has joined #openstack-infra | 14:18 | |
| openstackgerrit | Bob Ball proposed a change to openstack-infra/nodepool: Support nodes with launch condition https://review.openstack.org/65261 | 14:21 |
| openstackgerrit | Bob Ball proposed a change to openstack-infra/nodepool: Support install phase with nodepool https://review.openstack.org/61463 | 14:21 |
| openstackgerrit | Bob Ball proposed a change to openstack-infra/nodepool: Allow useage of server IDs as well as names. https://review.openstack.org/69424 | 14:21 |
| *** chandankumar has joined #openstack-infra | 14:21 | |
| *** mriedem has joined #openstack-infra | 14:23 | |
| *** emagana has quit IRC | 14:23 | |
| *** zz_ewindisch is now known as ewindisch | 14:23 | |
| *** dkliban has joined #openstack-infra | 14:23 | |
| *** yamahata has quit IRC | 14:25 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 14:25 | |
| openstackgerrit | godp1301 proposed a change to openstack-infra/jenkins-job-builder: Add best and worst status details for conditional-step plugin. https://review.openstack.org/69608 | 14:25 |
| *** b3nt_pin is now known as beagles | 14:26 | |
| *** DinaBelova is now known as DinaBelova_ | 14:26 | |
| *** yamahata has joined #openstack-infra | 14:29 | |
| *** ryanpetrello has joined #openstack-infra | 14:30 | |
| *** thuc has joined #openstack-infra | 14:31 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Split the config out into two files https://review.openstack.org/69503 | 14:32 |
| mordred | fungi: fixed your comments I believe | 14:33 |
| *** thuc_ has quit IRC | 14:35 | |
| *** chandankumar has quit IRC | 14:35 | |
| *** thuc has quit IRC | 14:36 | |
| *** senk has joined #openstack-infra | 14:36 | |
| *** DinaBelova_ is now known as DinaBelova | 14:37 | |
| *** gokrokve_ has quit IRC | 14:39 | |
| *** senk has quit IRC | 14:39 | |
| dansmith | sdague: can we finish this conversation about not pushing my set? | 14:40 |
| sdague | dansmith: sure | 14:40 |
| *** mestery_ has joined #openstack-infra | 14:40 | |
| sdague | so something early in your set is driving the load higher again, which is why large ops is falling over | 14:41 |
| sdague | because it's not falling over anywhere else right now | 14:41 |
| dansmith | that's not what was causing the failure the first time I pushed, | 14:41 |
| dansmith | and the second time I did, I had a fix for the issue it was hitting (which seems to have worked) | 14:41 |
| sdague | dansmith: ok, it's how it failed the 2 times it went yesterday | 14:41 |
| dansmith | *now* it may be load, although I don't really see how, since it's failing in init_host | 14:42 |
| dansmith | sdague: why do you say that? | 14:42 |
| dansmith | because you saw a timeout? | 14:42 |
| *** mestery__ has joined #openstack-infra | 14:42 | |
| sdague | dansmith: right | 14:42 |
| sdague | so "load higher" might not be the right base issue | 14:43 |
| *** gokrokve_ has joined #openstack-infra | 14:43 | |
| dansmith | sdague: but the timeout was because conductor had fallen over early during init, not because it was waiting too long | 14:43 |
| sdague | ok | 14:43 |
| dansmith | and I fixed that with a new patch in front | 14:43 |
| sdague | didn't that whole patch series fail again yesterday? | 14:43 |
| dansmith | no | 14:43 |
| sdague | after that | 14:43 |
| *** meste____ has joined #openstack-infra | 14:43 | |
| dansmith | sorry, after what? | 14:43 |
| *** mestery has quit IRC | 14:43 | |
| sdague | https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nova-network-objects,n,z | 14:44 |
| sdague | after you pushed again last night | 14:44 |
| dansmith | no, | 14:44 |
| *** senk has joined #openstack-infra | 14:44 | |
| dansmith | several passed in front | 14:44 |
| dansmith | everything failed after this: | 14:44 |
| dansmith | https://review.openstack.org/#/c/66634/ | 14:44 |
| dansmith | http://logs.openstack.org/34/66634/11/check/gate-tempest-dsvm-large-ops/726be53/logs/screen-n-net.txt.gz?level=TRACE | 14:45 |
| sdague | ok | 14:45 |
| dansmith | for that reason, | 14:45 |
| dansmith | which I don't understand yet, | 14:45 |
| dansmith | but I don't have any reason to believe that when that is resolved the others wouldn't pass, | 14:45 |
| dansmith | and splitting that set is non-trivial and significantly adds to the housekeeping to keep it all straight | 14:45 |
| sdague | ok | 14:45 |
| dansmith | so I just don't think it makes sense to not push the whole thing unless you have some other reason I don't understand | 14:45 |
| *** mestery_ has quit IRC | 14:46 | |
| sdague | so we probably have the resources right now, but that patch series basically consumes half of all available test nodes | 14:46 |
| russellb | also note this is a pre-req to revisiting the concurrency setting ... so don't want to slow it down if we can | 14:46 |
| *** mestery__ has quit IRC | 14:46 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 14:46 | |
| dansmith | sdague: seems like a scheduling/quota thing to work out | 14:46 |
| sdague | especially when the fail condition makes everything go to the timeout | 14:47 |
| russellb | sdague: could be a possible zuul improvement | 14:47 |
| dansmith | sdague: telling folks to arbitrarily limit the size of their queues doesn't seem like a good approach to me | 14:47 |
| russellb | pay attention to dependent patches in check, and rate limit the check jobs going through them, something like that | 14:47 |
| dansmith | russellb: or even throttle per user instead of per branch, but yeah | 14:48 |
| russellb | yeah per user may be easier, i dunno | 14:48 |
| sdague | yeh, don't know, right now all our zuul brain power is going into other things | 14:48 |
| *** senk has quit IRC | 14:49 | |
| sdague | dansmith: also realistically you are the outlier case :) | 14:49 |
| sdague | http://graphite.openstack.org/render/?from=-24hours&height=180&until=now&width=334&bgcolor=ffffff&fgcolor=000000&areaMode=stacked&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.building),%20%27Building%27),%20%27ffbf52%27)&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.ready),%20%27Available%27),%20%2700c868%27)&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.used),%20%27In%20Use%27),%20%276464ff%27)&ta | 14:49 |
| sdague | rget=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.delete),%20%27Deleting%27),%20%27c864ff%27)&title=Test%20Nodes&_t=0.38635385874658823 | 14:49 |
| dansmith | sdague: this was a tanget branch to immediately help the gate, and is blocking me from working on actual things I need to work on, so artificially making it harder/longer is really a problem, IMHO | 14:49 |
| *** senk has joined #openstack-infra | 14:49 | |
| sdague | dansmith: yeh, no worries, I was just trying to say could we trial balloon the patches just up to the first one that fails before pushing everything, grabbing a branch at that ref and pushing should be really not too bad | 14:50 |
| *** flaper87 is now known as flaper87|afk | 14:50 | |
| sdague | and can put the whole series back together real fast after that | 14:50 |
| dansmith | sdague: but then I have rebase pain if I make several changes to fix the problem and slap the others on top of it | 14:50 |
| dansmith | sdague: these are highly interdependent | 14:51 |
| *** jcoufal has quit IRC | 14:53 | |
| sdague | ok, just a suggestion | 14:53 |
| *** jcoufal has joined #openstack-infra | 14:54 | |
| *** prad has joined #openstack-infra | 14:55 | |
| *** ruhe is now known as _ruhe | 14:56 | |
| *** mestery has joined #openstack-infra | 14:57 | |
| *** hashar has joined #openstack-infra | 14:57 | |
| *** mestery_ has joined #openstack-infra | 14:58 | |
| *** chandankumar has joined #openstack-infra | 14:59 | |
| *** meste____ has quit IRC | 15:00 | |
| * mordred isn't REALLY following - but if I wanted to do what sdague was suggesting without screwing myself ... | 15:00 | |
| mordred | nope. nevermind. it sucks | 15:01 |
| * mordred likes working things out as he tries to actually type solutions :) | 15:01 | |
| sdague | heh | 15:01 |
| openstackgerrit | sebastian marcet proposed a change to openstack-infra/config: Implements: blueprint openid-oauth2-infra-implementation-documentation https://review.openstack.org/69620 | 15:01 |
| *** mestery__ has joined #openstack-infra | 15:01 | |
| *** ewindisch is now known as zz_ewindisch | 15:01 | |
| sdague | yeh, because of the new rax quota, we're actually kind of fine right now. Without that it would have starved the gate | 15:02 |
| *** mestery has quit IRC | 15:02 | |
| *** mrodden has quit IRC | 15:02 | |
| *** NithyaG has quit IRC | 15:02 | |
| *** jnoller has joined #openstack-infra | 15:03 | |
| *** NithyaG has joined #openstack-infra | 15:03 | |
| *** dcramer_ has joined #openstack-infra | 15:03 | |
| *** mestery_ has quit IRC | 15:03 | |
| *** jnoller has quit IRC | 15:04 | |
| *** jnoller_ has joined #openstack-infra | 15:04 | |
| *** IzikP_Radware has joined #openstack-infra | 15:04 | |
| *** weshay has joined #openstack-infra | 15:05 | |
| *** mestery has joined #openstack-infra | 15:05 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add rtfd trigger jobs for climate project https://review.openstack.org/68062 | 15:05 |
| *** chandankumar has quit IRC | 15:06 | |
| *** kraman has joined #openstack-infra | 15:06 | |
| *** mestery__ has quit IRC | 15:09 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Database fixture added https://review.openstack.org/69384 | 15:10 |
| *** rwsu has joined #openstack-infra | 15:12 | |
| markmc | dansmith, I totally agree with what you're saying, but you could push in batches if that would help | 15:14 |
| markmc | git push gerrit HEAD~30:refs/publish/master | 15:14 |
| markmc | then later after they've passed | 15:14 |
| markmc | git push gerrit HEAD~20:refs/publish/master | 15:14 |
| markmc | etc. | 15:14 |
| markmc | gerrit will handle that fine | 15:14 |
| dansmith | markmc: okay, I've never pushed to gerrit without git review | 15:15 |
| dansmith | markmc: if I don't have to break my local branch, then that's what I want | 15:15 |
| *** malini_afk is now known as malini | 15:16 | |
| markmc | yeah, so - git-review just pushes to refs/publish/master | 15:16 |
| markmc | HEAD~30:refs/publish/master is a refspec | 15:16 |
| dansmith | yeah | 15:16 |
| *** mrodden has joined #openstack-infra | 15:16 | |
| markmc | git-push manpage has a nice explanation of refspecs | 15:16 |
| *** coolsvap has joined #openstack-infra | 15:16 | |
| markmc | but that means, "everything up to 30 patches back on this branch" | 15:17 |
| dansmith | yeah, I'm familiar, I'm just trained to not have to know the mechanics of how my patches get there :) | 15:17 |
| markmc | ok | 15:17 |
| mordred | yay tools! | 15:17 |
| *** zz_ewindisch is now known as ewindisch | 15:19 | |
| *** jgrimm has joined #openstack-infra | 15:20 | |
| *** che-arne has quit IRC | 15:20 | |
| *** rcleere has joined #openstack-infra | 15:22 | |
| *** morganfainberg|z is now known as morganfainberg | 15:22 | |
| *** dkranz has joined #openstack-infra | 15:24 | |
| *** thedodd has joined #openstack-infra | 15:26 | |
| *** talluri has quit IRC | 15:28 | |
| *** talluri has joined #openstack-infra | 15:28 | |
| *** _ruhe is now known as ruhe | 15:30 | |
| *** yaguang has quit IRC | 15:30 | |
| *** senk has quit IRC | 15:30 | |
| *** yamahata has quit IRC | 15:30 | |
| ttx | oops. | 15:30 |
| ttx | mordred: I was cleaning up milestone-proposed branches from icehouse-2 and I accidentally removed swift's milestone-proposed branch as well (the yet-to-be-released one for 1.12.0) | 15:31 |
| ttx | mordred: is there any convenient way to restore that ? | 15:31 |
| *** talluri has quit IRC | 15:32 | |
| ttx | notmyname: worst case scenario I can recreate it and we can repush the backport | 15:35 |
| *** senk has joined #openstack-infra | 15:36 | |
| ttx | fungi: ^if you have any suggestions... I think we are stuck because it wasn't tagged yet | 15:36 |
| mordred | ttx: do you know what the sha was supposed to have been? | 15:41 |
| *** eharney has quit IRC | 15:41 | |
| ttx | looks like https://review.openstack.org/gitweb?p=openstack%2Fswift.git;a=commitdiff;h=78704010cc4c3b5c2bd050ee846323506ad3ed5b | 15:41 |
| *** david-lyle has joined #openstack-infra | 15:41 | |
| ttx | Was about to try to recreate at 78704010cc4c3b5c2bd050ee846323506ad3ed5b | 15:41 |
| mordred | ttx: I think recreate is the right choice | 15:41 |
| ttx | will download the tarball first to check if it generates the same content | 15:42 |
| *** wenlock has joined #openstack-infra | 15:43 | |
| ttx | OK recreated | 15:43 |
| ttx | man, that git thing is really magic | 15:43 |
| ttx | notmyname: branch recreated, just doublecheck it's sane | 15:44 |
| ArxCruz | ALL: Can someone help me with zuul pipelines ? I want to report back my results, but I don't want to vote +1 or -1 (for now). If someone have an example, that would be awesome :) | 15:45 |
| mordred | ArxCruz: one sec - I'll paste a link | 15:46 |
| ArxCruz | mordred: thanks! | 15:46 |
| dstanek | hi all, how do we go about getting a non-voting python3 job for keystone? | 15:46 |
| mordred | ArxCruz: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n234 | 15:47 |
| *** amotoki has joined #openstack-infra | 15:47 | |
| ArxCruz | mordred: okay, but which pipeline should I add in my yaml to post the results ? | 15:48 |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 15:48 | |
| dstanek | i was looking for our jenkins-job-builder yaml, but i have no idea where to find it | 15:50 |
| *** jcoufal has quit IRC | 15:50 | |
| *** jcoufal-mobile has joined #openstack-infra | 15:51 | |
| *** andreaf has quit IRC | 15:52 | |
| ArxCruz | dstanek: it is on zuul yaml file :) | 15:52 |
| ArxCruz | dstanek: by the way, I'm looking for some zuul example to report back (with no voting) | 15:53 |
| dstanek | ArxCruz: is that in openstack-infra/config somewhere? | 15:54 |
| *** rcleere has quit IRC | 15:54 | |
| ArxCruz | dstanek: just a sec, let me check | 15:54 |
| ArxCruz | dstanek: same url mordred points me | 15:55 |
| ArxCruz | http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n234 | 15:55 |
| *** reed has joined #openstack-infra | 15:55 | |
| dstanek | mordred: thx - i missed that | 15:56 |
| * fungi is back but on a conference call for the next hour, so not really paying attention yet | 15:57 | |
| *** rcleere has joined #openstack-infra | 15:57 | |
| *** wenlock_ has joined #openstack-infra | 15:58 | |
| *** AlexF_ has joined #openstack-infra | 16:01 | |
| *** freyes has quit IRC | 16:01 | |
| *** starmer has joined #openstack-infra | 16:02 | |
| openstackgerrit | sebastian marcet proposed a change to openstack-infra/config: Implements: blueprint openid-oauth2-infra-implementation-puppet-script [smarcet] - #5478 - Clean up puppet Scripts https://review.openstack.org/69636 | 16:04 |
| notmyname | ttx: lgtm | 16:04 |
| ttx | notmyname: checked that the tarball content was identical too | 16:05 |
| *** hashar has quit IRC | 16:05 | |
| ttx | sorry for the glitch. | 16:05 |
| mordred | smarcet: hey! I don't know much about your codebase yet - but quick question - two different mysql hosts and a redis? | 16:06 |
| fungi | mordred: apparently one db is the silverstripe backend, the other is the openid provider backend | 16:07 |
| *** hashar has joined #openstack-infra | 16:07 | |
| smarcet | @mordred: redis host will run most likelly locally, but mysql hosts will be external ones and would be probably hosted on trove | 16:07 |
| *** boris-42 has quit IRC | 16:07 | |
| mordred | smarcet, fungi: ah - ok. makes sense | 16:07 |
| mordred | also, what's a greggilbert? :) | 16:07 |
| smarcet | fungi: that is rite, one mysql for openstackid and another one for ss db | 16:08 |
| fungi | in theory, the databases might get combined in the future once the org site gets opened up | 16:08 |
| smarcet | mordred: is a laravel lib that provides recaptcha implementation | 16:08 |
| mordred | kk | 16:08 |
| *** jcooley_ has joined #openstack-infra | 16:09 | |
| *** mfer has joined #openstack-infra | 16:09 | |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 16:09 | |
| *** eharney has joined #openstack-infra | 16:09 | |
| smarcet | mordred:https://packagist.org/packages/greggilbert/recaptcha | 16:10 |
| *** flaper87|afk is now known as flaper87 | 16:12 | |
| *** lcheng has joined #openstack-infra | 16:12 | |
| *** virmitio has joined #openstack-infra | 16:12 | |
| *** pcrews has joined #openstack-infra | 16:13 | |
| *** emagana has joined #openstack-infra | 16:14 | |
| *** DinaBelova is now known as DinaBelova_ | 16:14 | |
| *** jcoufal-mobile has quit IRC | 16:15 | |
| *** jcoufal-mobile has joined #openstack-infra | 16:15 | |
| *** che-arne has joined #openstack-infra | 16:16 | |
| *** emagana has quit IRC | 16:16 | |
| *** atiwari has joined #openstack-infra | 16:18 | |
| *** SumitNaiksatam has quit IRC | 16:18 | |
| *** vkozhukalov has quit IRC | 16:19 | |
| *** VijayT has joined #openstack-infra | 16:19 | |
| openstackgerrit | sebastian marcet proposed a change to openstack-infra/config: Implements: blueprint openid-oauth2-infra-implementation-puppet-script [smarcet] - #5478 - Clean up puppet Scripts https://review.openstack.org/69636 | 16:22 |
| *** jcoufal has joined #openstack-infra | 16:23 | |
| *** jcoufal-mobile has quit IRC | 16:26 | |
| *** thuc has joined #openstack-infra | 16:27 | |
| *** talluri has joined #openstack-infra | 16:28 | |
| *** thuc has quit IRC | 16:28 | |
| *** thuc has joined #openstack-infra | 16:28 | |
| *** thuc has quit IRC | 16:28 | |
| *** gyee has joined #openstack-infra | 16:29 | |
| *** thuc has joined #openstack-infra | 16:29 | |
| *** thuc has quit IRC | 16:29 | |
| *** thuc has joined #openstack-infra | 16:30 | |
| *** wenlock has quit IRC | 16:30 | |
| *** thuc has quit IRC | 16:30 | |
| portante | any logstash kibana query experts around? | 16:31 |
| *** ruhe is now known as _ruhe | 16:31 | |
| *** thuc has joined #openstack-infra | 16:31 | |
| portante | I am trying to understand why for this effort, https://review.openstack.org/69068, I am not able to reduce the double hits to one. | 16:32 |
| portante | this is for the uncat horizon bugs (top of the uncat page) | 16:32 |
| *** dizquierdo has quit IRC | 16:32 | |
| jesusaurus | what do you mean by double hits? | 16:33 |
| *** SumitNaiksatam has joined #openstack-infra | 16:33 | |
| portante | it is reporting the same log file twice in the results | 16:38 |
| sdague | portante: that's ok | 16:38 |
| sdague | we do squashing during the reporting for failures in the overall hit page | 16:39 |
| portante | okay, so then we can just take that commit as is then? | 16:39 |
| portante | sdague: see the comments on the commit | 16:39 |
| portante | sdague: I thought the posted query would reduce the double hits, but it does not match anything, which says I really don't understand what the query is doing | 16:40 |
| sdague | portante: yeh, I think jog0 just wanted to figure out if there was a narrower one | 16:40 |
| openstackgerrit | David Stanek proposed a change to openstack-infra/config: Adds a non-voting python33 check to Keystone https://review.openstack.org/69649 | 16:40 |
| sdague | portante: where is the other proposed query? | 16:41 |
| portante | in the last comment | 16:41 |
| dstanek | mordred, ArxCruz: is this really all i have to do? https://review.openstack.org/#/c/69649/ | 16:41 |
| portante | I justed added another message clause for one of the tests that failed, which is in all the logs that I have sampled | 16:42 |
| sdague | portante: so you can only match single log lines, because of the way things are indexed | 16:42 |
| sdague | which is how logstash works | 16:42 |
| ArxCruz | dstanek: according mordred told me, yes | 16:42 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1268631 https://review.openstack.org/69068 | 16:43 |
| sdague | which is less than ideal, but we've gotten a lot of mileage on it so far, so we haven't implemented multiline matches yet | 16:43 |
| portante | meaning, the two "message" clauses have to match on the same line? | 16:43 |
| sdague | portante: yes | 16:44 |
| sdague | because each line is a document to logstash | 16:44 |
| portante | k, then I don't see a way to narrow this one down further | 16:44 |
| sdague | yep, agree | 16:45 |
| portante | thanks, I think I get it now | 16:45 |
| sdague | I just approved it, it's in the repo now | 16:45 |
| sdague | thanks | 16:45 |
| portante | great, thanks | 16:45 |
| *** chandankumar has joined #openstack-infra | 16:45 | |
| dstanek | i don't see a lot of pypy checks - is that something worth doing for projects? | 16:45 |
| dstanek | ArxCruz: thx | 16:46 |
| ArxCruz | np | 16:46 |
| *** lcheng_ has joined #openstack-infra | 16:47 | |
| *** hashar has quit IRC | 16:50 | |
| *** lcheng has quit IRC | 16:50 | |
| *** hashar has joined #openstack-infra | 16:51 | |
| *** whoops has joined #openstack-infra | 16:51 | |
| clarkb | morning | 16:56 |
| *** emagana has joined #openstack-infra | 16:56 | |
| anteaya | morning clarkb | 16:58 |
| zaro | morning | 16:58 |
| *** DinaBelova_ is now known as DinaBelova | 17:00 | |
| *** afazekas has quit IRC | 17:00 | |
| zaro | clarkb, mordred : found this issue with mysql and jeepyb. simple fix, probably should go in soon.. https://review.openstack.org/#/c/69543/ | 17:00 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add trove translation jobs https://review.openstack.org/64309 | 17:02 |
| clarkb | I am just going to approve that, it is a merge conflict resolution for a change approved yesterday | 17:02 |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 17:02 | |
| mordred | kk | 17:02 |
| clarkb | zaro: is that tuple compatible with older versions too? | 17:02 |
| notmyname | mordred: swift 1.12.0 was tagged on m-p proposed today. can you check that the merge job will happen so that the versioning keeps working? | 17:03 |
| *** e0ne_ has quit IRC | 17:03 | |
| zaro | clarkb: it works with MySQL-python version 1.2.4 and 1.2.5 | 17:03 |
| *** marun has joined #openstack-infra | 17:03 | |
| mordred | notmyname: it ran and failed... looking | 17:04 |
| *** e0ne has joined #openstack-infra | 17:04 | |
| clarkb | zaro: perfect thanks, approved | 17:04 |
| notmyname | mordred: thanks | 17:04 |
| *** marun has quit IRC | 17:05 | |
| *** dcramer_ has quit IRC | 17:07 | |
| anteaya | morning zaro | 17:07 |
| *** boris-42 has joined #openstack-infra | 17:07 | |
| *** kraman has quit IRC | 17:08 | |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: fix manage_projects with MySQL-Python 1.2.5 https://review.openstack.org/69543 | 17:08 |
| *** marun has joined #openstack-infra | 17:08 | |
| *** e0ne has quit IRC | 17:09 | |
| mordred | notmyname: https://review.openstack.org/69666 | 17:09 |
| *** chandankumar has quit IRC | 17:09 | |
| mordred | notmyname: I ran the script locally to do that, and it worked, so I'm going to need to investigate further why it was unhappy on the jenkins slave | 17:09 |
| *** kraman has joined #openstack-infra | 17:10 | |
| notmyname | mordred: interesting. (I thought) I had resolved that conflict when I backported the last patch onto m-p. probably the merge back to master is causing the issue | 17:11 |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 17:11 | |
| mordred | notmyname: it's _supposed_ to null-merge | 17:12 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Run the tests for the OpenStack SDK tests https://review.openstack.org/69667 | 17:12 |
| vjay | Hi, I am new to OpenStack infra. Jenkins reported failures due to bugs which are listed in http://status.openstack.org/rechecks/ in Gerrit. But there is no entry from Elastic Recheck in Gerrit. | 17:14 |
| *** starmer has quit IRC | 17:14 | |
| vjay | What could be the reason? | 17:14 |
| anteaya | vjay: elastic recheck doesn't know about the bug | 17:14 |
| anteaya | you can run recheck bug #bug number | 17:14 |
| anteaya | you also can add the bug number to elastic recheck | 17:15 |
| *** _ruhe is now known as ruhe | 17:15 | |
| anteaya | vjay: http://docs.openstack.org/infra/elastic-recheck/readme.html | 17:15 |
| *** chandankumar has joined #openstack-infra | 17:15 | |
| anteaya | thanks for asking and welcome to -infra | 17:15 |
| vjay | let me check | 17:16 |
| *** gokrokve_ has quit IRC | 17:16 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Correct .Fl typo WRT --compare in the manual page https://review.openstack.org/66182 | 17:16 |
| notmyname | mordred: I've got to drive to San Jose now. I'll check in again in an hour or so | 17:16 |
| *** pcrews has quit IRC | 17:17 | |
| * hashar teleports notmyname | 17:18 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools gates for openstack-manuals https://review.openstack.org/69669 | 17:19 |
| openstackgerrit | Peter Portante proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1266616 https://review.openstack.org/69670 | 17:20 |
| smarcet | fungi: mordred: hi guys, i am getting a puppet syntax error here https://review.openstack.org/#/c/69636/ | 17:20 |
| smarcet | but http://logs.openstack.org/36/69636/2/check/gate-config-puppet-syntax/b8954d4/console.html is referring to files that were not included on my patch , like | 17:20 |
| smarcet | [0;33mwarning: Deprecation notice: Resource references should now be capitalized on line 557 in file /home/jenkins/workspace/gate-config-puppet-syntax/modules/gerrit/manifests/init.pp | 17:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Add trove translation jobs https://review.openstack.org/64309 | 17:21 |
| *** moted has joined #openstack-infra | 17:21 | |
| fungi | smarcet: i bet we started testing with a new puppet-lint release | 17:21 |
| * fungi checks | 17:22 | |
| *** dkliban is now known as dkliban_afk | 17:23 | |
| fungi | smarcet: oh, those are only warnings... i forgot we don't currently fail on warnings, just errors | 17:25 |
| vjay | anteaya: thanks for the super quick response. Let me see if I can understand. once i have commented "recheck bug #bug_number", any further patchset submission failures due to the same reason/query will be identified by Elastic Search. | 17:25 |
| fungi | smarcet: if you scroll down to the end of the log, you'll see some syntax errors in modules/redis/templates/redis.2.4.conf.erb | 17:27 |
| smarcet | fungi: ok i think that i know what the error is , tnx :) | 17:27 |
| clarkb | mordred: if I address jeblair's comment in https://review.openstack.org/#/c/62187/ and get the beginning of that stack merged then the jeepyb change can land? | 17:27 |
| smarcet | yeah just saw it, i overlooked it :( | 17:27 |
| anteaya | vjay: not quite | 17:27 |
| *** zhiyan has quit IRC | 17:27 | |
| fungi | smarcet: it's like looking for a needle in a haystack sometimes | 17:27 |
| *** ruhe is now known as _ruhe | 17:28 | |
| anteaya | elastic recheck bug numbers are submitted by hand: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/queries | 17:28 |
| *** flaper87 is now known as flaper87|afk | 17:28 | |
| jnoller_ | sdague: did you get all set up? | 17:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Adds a non-voting python33 check to Keystone https://review.openstack.org/69649 | 17:28 |
| clarkb | woah ^ is exciting | 17:29 |
| clarkb | did keystoneclient ever get py33 sorted? | 17:29 |
| anteaya | in order for elastic recheck to know about a bug, some wonderful contributor has taken the time to identify a logstash fingerprint for that bug and submit it as a commit to the elastic-recheck repo | 17:30 |
| fungi | clarkb: i figured that one was safe enough to only need one core review | 17:30 |
| mordred | clarkb: other way around | 17:30 |
| mordred | clarkb: we need to merge the jeepyb change, then we can land that change | 17:31 |
| mordred | the jeepyb change is backwards compatible | 17:31 |
| *** BobBall is now known as BobBallAway | 17:31 | |
| vjay | probably i should rephrase my question. In my case, there is already a bug and already a yaml submitted. Elastic Recheck entry is not adding an entry to my gerrit review submit. | 17:31 |
| anteaya | vjay: using recheck no bug allows recheck to know about it, and also is the syntax zuul is looking for to all ow the patchset to be testing in the check queue again | 17:31 |
| anteaya | vjay: let's get more specific | 17:32 |
| anteaya | please post the url of your patch, the bug and the yaml file in question | 17:32 |
| anteaya | there might be an error in syntax | 17:32 |
| clarkb | mordred: jeepyb is backward compat but puppet isn't aiui | 17:32 |
| mordred | right | 17:32 |
| vjay | Patch: https://review.openstack.org/#/c/57524 is my code submit | 17:32 |
| clarkb | or are they both compatible in both directions? | 17:33 |
| mordred | so we want to land the jeepyb change, which will understand old and new format | 17:33 |
| mordred | then we can land the puppet change, which will move the file to the new format | 17:33 |
| clarkb | mordred: gotcha | 17:33 |
| clarkb | mordred: I am about to push to that stack addressing jeblair's comments | 17:33 |
| openstackgerrit | sebastian marcet proposed a change to openstack-infra/config: Implements: blueprint openid-oauth2-infra-implementation-puppet-script [smarcet] - #5478 - Clean up puppet Scripts https://review.openstack.org/69636 | 17:33 |
| *** chandankumar has quit IRC | 17:34 | |
| *** yassine has quit IRC | 17:34 | |
| anteaya | vjay: who is embrance ci? | 17:35 |
| vjay | yaml: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/diff/queries/1254890.yaml | 17:35 |
| vjay | i dont know :-) | 17:35 |
| pleia2 | kashyap: left some notes in the etherpad re: fedora in the gate | 17:35 |
| clarkb | pleia2: thanks | 17:36 |
| anteaya | back in conf tutorial, I'll come back at next break unless someone helps you before that | 17:36 |
| fungi | anteaya: embrane ci is one of the third-party test systems... for neutron i think | 17:36 |
| *** SergeyLukjanov_ is now known as SergeyLukjanov | 17:36 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add Storyboard puppet module https://review.openstack.org/65017 | 17:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Unlaunchpadify projects.yaml https://review.openstack.org/62189 | 17:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add storyboard-dev node with krotscheck and ruhe on it https://review.openstack.org/69516 | 17:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Track direct-release projects in projects.yaml https://review.openstack.org/62190 | 17:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Split config from projects list https://review.openstack.org/62187 | 17:36 |
| vjay | bug: https://bugs.launchpad.net/nova/+bug/1254890 | 17:36 |
| clarkb | fungi: any chance you can rereview the jeepyb change for ini config then we can review the stack above? | 17:36 |
| fungi | anteaya: i believe it went haywire a while back and we revoked voting for it, now the operators of it are waiting on markmcclain to okay allowing it to vote again | 17:37 |
| kashyap | pleia2, Thank you, will check tomorrow. It's late here :-) | 17:37 |
| fungi | clarkb: i'm sure the chance is nonzero. i'll go find it now | 17:37 |
| dims | fungi, when you get a chance, please consider blessing this - https://review.openstack.org/#/c/67508/ ("Add jenkins master name to the logstash records") | 17:37 |
| anteaya | fungi: okay thanks, later I will checkin with markmcclain about it | 17:37 |
| clarkb | fungi: trying to kill as many changes in review as possible :) | 17:38 |
| clarkb | dims: I can bless that | 17:38 |
| dims | thanks! | 17:38 |
| clarkb | dims: would it be Ok to make that a WIP until we have the plugin installed everywhere? | 17:38 |
| vjay | The failure log in : http://logs.openstack.org/24/57524/18/check/check-tempest-dsvm-neutron-pg/1a5c336/logs/testr_results.html.gz is matching http://git.openstack.org/cgit/openstack-infra/elastic-recheck/diff/queries/1254890.yaml | 17:39 |
| dims | clarkb, code logs "UNKNOWN" if data is not available. but am ok to WIP as well | 17:40 |
| clarkb | dims: I suppose we can merge it then get the data when available | 17:40 |
| dims | clarkb, sounds good | 17:40 |
| *** dcramer_ has joined #openstack-infra | 17:42 | |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Split the config out into two files https://review.openstack.org/69503 | 17:42 |
| ArxCruz | ALL: I have my nodepool server running, it's adding the VM's on jenkins, jenkins is running tempest fine, however, after the tempest job finishes, the VM isn't automatically removed and a new one is added, it continues to try run other jobs in queue, and then it fails (due the garbage left from the last run) | 17:43 |
| clarkb | ArxCruz: in our jobs list in layout.yaml for zuul we have a single use server function applied to *dsvm* jobs. I think you need that | 17:44 |
| *** ICmonitor1 has joined #openstack-infra | 17:44 | |
| clarkb | that function tells gearman plugin on the jenkins side to mark the node offline when the test begins so that only one test will run on it | 17:44 |
| *** _david_ has joined #openstack-infra | 17:44 | |
| ArxCruz | clarkb: you know the function name ? | 17:44 |
| _david_ | zaro, which plugin? | 17:45 |
| fungi | ArxCruz: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n198 | 17:45 |
| ArxCruz | fungi: thanks! | 17:45 |
| fungi | ArxCruz: "parameter-function: single_use_node" | 17:45 |
| *** gokrokve has joined #openstack-infra | 17:45 | |
| jerryz | clarkb: sometimes, i still got confused. i used to have the same condition as ArxCruz, and i add the single use option in layout.yaml. But aren't the slaves labeled 'used' once a job ran on it? even it didn't get deleted, new jobs are not supposed to run a used slave in the first place. | 17:46 |
| *** bookwar has quit IRC | 17:46 | |
| clarkb | jerryz: they are marked used in the nodepool DB but not in jenkins | 17:46 |
| zaro | _david_: replication plugin | 17:46 |
| *** ICmonitor has quit IRC | 17:47 | |
| clarkb | so jenkins will happily run more jobs on the slave | 17:47 |
| *** malini is now known as malini_afk | 17:47 | |
| *** hashar has quit IRC | 17:47 | |
| zaro | _david_: https://groups.google.com/forum/#!topic/repo-discuss/vgDzcPjVmJ8 | 17:47 |
| _david_ | zaro this is core plugin | 17:47 |
| jerryz | clarkb: so nodepool won't change the label in jenkins as d-g jobs used to do | 17:47 |
| *** sandywalsh has quit IRC | 17:47 | |
| clarkb | jerryz: I don't think so | 17:47 |
| *** gokrokve has quit IRC | 17:47 | |
| clarkb | jerryz: and even if it does that code was always racy | 17:47 |
| fungi | jerryz: the label changing had unfortunate race conditions | 17:47 |
| zaro | _david_: ohh yeah, forgot. but generally should 3rd party plugin bugs go into main gerrit issue tracker? | 17:48 |
| fungi | jerryz: jenkins tended not to process the relabel request right away | 17:48 |
| _david_ | zaro, i would say yes. | 17:48 |
| fungi | jerryz: the jenkins-gearman plugin now has functionality to offline the nodes in jenkins instead | 17:48 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools gates for openstack-manuals https://review.openstack.org/69669 | 17:48 |
| _david_ | (main project) | 17:48 |
| _david_ | zaro, i would say you can't do that with alias., but why you ned to? | 17:49 |
| _david_ | zaro, what is wrong with ssh gerrit replication start? | 17:49 |
| _david_ | zaro, why adapt Gerrit and not your script? | 17:49 |
| zaro | _david_: from the documentation i thought that's what the ssh-alias allows you to do. | 17:50 |
| _david_ | zaro, yeas, but inside core | 17:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Add jenkins master name to the logstash records https://review.openstack.org/67508 | 17:50 |
| _david_ | zaro, not between core and plugins, because namespaces are different, (i think) | 17:51 |
| jerryz | fungi: great. the gearman plugin is updated? | 17:51 |
| *** ICmonitor1 has quit IRC | 17:51 | |
| zaro | _david_: i see. i guess we only have option then? | 17:51 |
| _david_ | zaro, can't parse your last statement? | 17:52 |
| zaro | _david_: i mean *one* option | 17:52 |
| _david_ | zaro, still don get what is wrong with ssh gerrit replication start ? | 17:52 |
| zaro | _david_: nothing, except that it breaks backwards compatability with stuff that depends on the gerrit replicate command | 17:53 |
| jerryz | fungi: oh, i see it is a feature in 0.0.5 | 17:53 |
| _david_ | zaro, what backwards compatibility? why it is needed? If we migrate to 2.8 who cares about backwards compatibility? | 17:53 |
| zaro | _david_: i would prefer to not change our scripts | 17:53 |
| clarkb | fungi: thank you for reviewing the jeepyb split config change. The companions to that change start at https://review.openstack.org/#/c/62187/ | 17:53 |
| fungi | zaro: _david_: i think it's probably fine if we update our manage-projects script to start doing a version check through the api and then branch on which commands it uses depending on the gerrit release it's talking to, assuming gerrit upstream is not concerned with api backward compatibility | 17:54 |
| clarkb | _david_: zaro: gerritlib is a gerritlib. not a gerrit 2.4 lib or a gerrit 2.8 lib | 17:54 |
| clarkb | it needs to be backward compat | 17:54 |
| fungi | oh, right, we extracted the replication call out to gerritlib | 17:54 |
| clarkb | _david_: it is insane that a command name like that which honestly doesn't matter as long as it stays the same would change | 17:55 |
| fungi | so gerritlib should probably start doing version detection if there are no forward/backward compat guarantees for the gerrit api | 17:55 |
| clarkb | that said, its not like we worry terribly much about backward compat around here | 17:55 |
| zaro | i'm looking into adding version command to gerritlib now | 17:55 |
| _david_ | clarkb, I still don't understand that backward compatibility issue you mentioned with gerritlib | 17:56 |
| *** gokrokve has joined #openstack-infra | 17:56 | |
| fungi | right, i think version detection is likely to become necessary anyway since this is probably one of many breaking api changes we should expect from future gerrit releases | 17:56 |
| clarkb | gerrit seems to have realized this problem and added ssh aliases | 17:56 |
| portante | sdaque: I am looking at the gate-grenade-dsvm uncat jobs | 17:56 |
| portante | sdague: sorry | 17:56 |
| clarkb | _david_: we maintain python's gerritlib on pypi. That library should be able to talk to different versions without its users carring what the command is | 17:56 |
| zaro | ohh yeah, i also patched python-jenkins with version command. jeblair can you do your magic on that so we could use in jjb? | 17:57 |
| clarkb | _david_: it adds a bit of complexity to the lib to now need to do version detection and command lookups in a table | 17:57 |
| portante | there is a high correlation between the "iSCSI device not found" errors in logs and about 6+ of those jobs failing | 17:57 |
| *** ewindisch is now known as zz_ewindisch | 17:57 | |
| portante | but I am having a hard time finding the bug to hang that off of | 17:57 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add missed climate repos https://review.openstack.org/69680 | 17:58 |
| *** IzikP_Radware has quit IRC | 17:58 | |
| *** zz_ewindisch is now known as ewindisch | 17:58 | |
| zaro | clarkb, _david_ : just thought i could alias the command instead. i'll just patch our script. | 17:58 |
| _david_ | clarkb, KISS principle. Replication feature was extracted to a plugin 4 years ago or something. I am not aware of any Gerrit installation that has 2.4 version. So just change it and forget the issue with version detection | 17:58 |
| clarkb | _david_: it wasn't 4 years ago.. I have only been working on this stuff for less than 2 years :P | 17:59 |
| _david_ | clarkb, ,-) | 17:59 |
| clarkb | _david_: the problem isn't that it became a plugin it is that the command name itself changed | 17:59 |
| zaro | wasn't it since 2.6? | 17:59 |
| clarkb | what was wrong with `gerrit replicate` | 17:59 |
| clarkb | now it is `replications start` | 17:59 |
| *** e0ne has joined #openstack-infra | 17:59 | |
| _david_ | clarkb, i try to explain: the command name didn't change at all ... | 17:59 |
| _david_ | clarkb, it was start and is remained start | 18:00 |
| *** gokrokve has quit IRC | 18:00 | |
| _david_ | replication is the plugin name | 18:00 |
| _david_ | ro prevent clashes between different plugin an core | 18:00 |
| fungi | _david_: well, we're not upgrading overnight. we need to be able to test the new version of gerrit with our library while still supporting the version we have in production, at least until we've safely worked out all the changes we need to implement to upgrade and get it to a point where we know we don't need to roll back. so i think version detection or at least some fallback (try one way, then try | 18:00 |
| fungi | the other when it fails) will be necessary for at least a transitional period | 18:00 |
| _david_ | so you can have plugin foo and bar and both can have SSH command baz | 18:00 |
| _david_ | ssh gerrit foo baz | 18:01 |
| _david_ | ssh gerrit bar baz | 18:01 |
| clarkb | _david_: maybe the alias needs to be installed by default then? | 18:01 |
| clarkb | anyways the ship has sailed, we will deal with it | 18:01 |
| _david_ | clarkb, fungi zaro WIP plugin idea: | 18:02 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add murano with all subprojects https://review.openstack.org/69681 | 18:02 |
| _david_ | i have an idea how we can my be not need that core change that was rejected: | 18:02 |
| _david_ | Just use DRAFT status as replacement for WORKINPROGRESS status, and block draft workflow completely. | 18:03 |
| _david_ | i still have to veriy it | 18:03 |
| clarkb | interesting | 18:03 |
| *** sandywalsh has joined #openstack-infra | 18:03 | |
| clarkb | so overload that status and make the behavor different? | 18:03 |
| _david_ | If it is working, we are done | 18:03 |
| _david_ | clarkb, exactly | 18:03 |
| _david_ | clarkb, https://gerrit-review.googlesource.com/#/c/54062 | 18:04 |
| fungi | that's certainly an idea. as long as we can still control draft/undraft as non-change-owners via acl | 18:04 |
| zaro | thought draft feature was going to get reworked? | 18:04 |
| _david_ | in this change i disabled pushing to refs/drafts | 18:05 |
| *** DinaBelova is now known as DinaBelova_ | 18:05 | |
| fungi | also it would be nice to still be able to draft/undraft through the api too (we want to make git-review capable of submitting changes and putting them into wip status immediately without button clicking in the webui) | 18:06 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools gates for openstack-manuals https://review.openstack.org/69669 | 18:07 |
| _david_ | fungi, shoudn't it already doable with ssh command? | 18:07 |
| clarkb | mordred: still around? https://review.openstack.org/#/c/68439/ when you bumped RAM quota did you bump cpu quota too? | 18:08 |
| *** pcrews has joined #openstack-infra | 18:08 | |
| clarkb | mordred: there were cpu quota limit log error messages in the nodepool logs yesterday | 18:08 |
| zaro | _david_: draft/undraft is not part of draft workflow | 18:09 |
| zaro | _david_: how can you do it thru ssh? | 18:09 |
| _david_ | zaro, not draft/undraft, but wip/ready | 18:10 |
| mordred | clarkb: I have no idea. | 18:10 |
| _david_ | zaro, https://github.com/davido/gerrit-wip-plugin/blob/master/src/main/java/com/googlesource/gerrit/plugins/wip/SetCommand.java | 18:11 |
| mordred | clarkb: I just asked pvo for quota bump and he made magic happen | 18:11 |
| fungi | _david_: yes, our old wip patch made api extensions, but we ended up not implementing them long-term in git-review because we expected the implementation to change in later gerrit anyway | 18:11 |
| fungi | mordred: vcpu quota? | 18:11 |
| _david_ | fungi, in our plugin, where i have +2 we can do everything what you like ;-) | 18:11 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add mistral with subprojects https://review.openstack.org/69683 | 18:12 |
| fungi | _david_: right. assuming that doesn't change when switching to overloading the draft status field to track it, then it sounds like precisely the interface we should implement in git-review | 18:13 |
| mordred | fungi: I asked for it in terms of numbers of 8G instances | 18:13 |
| fungi | mordred: that's a sane enough choice (we don't have many which aren't, and those are pretty much all in dfw) | 18:13 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add murano with all subprojects https://review.openstack.org/69681 | 18:14 |
| zaro | _david_: cool. it's an idea worth exploring, however my concern is that draft feature might drastically changed so i don't know the impact on depending on it. | 18:14 |
| *** dkliban_afk is now known as dkliban | 18:14 | |
| *** praneshp has joined #openstack-infra | 18:15 | |
| clarkb | AaronGr: https://review.openstack.org/#/c/66072/2 did you plan on responding to jeblair there? | 18:15 |
| *** mkerrin has left #openstack-infra | 18:15 | |
| _david_ | zaro, we just turn it off: nothing simpler as that, not matter how it is going to change, it is turn off and overloaded: | 18:16 |
| AaronGr | clarkb: i did .. but i wonder why it didn't save my change. will re-add | 18:16 |
| clarkb | AaronGr: thanks | 18:16 |
| clarkb | fungi: the other change (if I can be selfish) that I think could use review is https://review.openstack.org/#/c/69442/. That should simplify our use of puppet for slaves, fix sudo on single use unittest nodes, and fix the dpkg thing | 18:17 |
| AaronGr | clarkb: i see, it's showing as a draft. how do i un-draft a comment? | 18:17 |
| _david_ | zaro, fungi https://gerrit-review.googlesource.com/#/c/54075 , i will verify this approach and let you know | 18:17 |
| clarkb | AaronGr: go to the top change page, and click review | 18:17 |
| *** harlowja_away is now known as harlowja | 18:17 | |
| clarkb | then save the review. It will give you the option for a cover message and show you your inline messages | 18:17 |
| zaro | _david_: cool. i will review that sometime soon. | 18:18 |
| zaro | _david_: btw, dborowitz says gerrit issue tracker should only be for core plugins. | 18:18 |
| *** gokrokve has joined #openstack-infra | 18:18 | |
| _david_ | zaro, ok | 18:19 |
| fungi | clarkb: yeah, i started reviewing 69442 but had to take a break. it was brain-twisty. trying again in a bit | 18:20 |
| *** gokrokve_ has joined #openstack-infra | 18:20 | |
| _david_ | clarkb, zaro, fungi folks want to know details about Zuul: https://groups.google.com/forum/#!topic/repo-discuss/Wica7iJ5DGk | 18:20 |
| *** _david_ has quit IRC | 18:20 | |
| AaronGr | clarkb: thanks | 18:21 |
| clarkb | fungi: np I spent a good 10 minutes or so building a model of includes in my head | 18:21 |
| fungi | it looks well-thought out. i just need to be able to repeat most of the well-thinking ;) | 18:21 |
| clarkb | we had managed to cthulu it a bit | 18:21 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools gates for openstack-manuals https://review.openstack.org/69669 | 18:21 |
| *** VijayT has quit IRC | 18:22 | |
| *** gokrokv__ has joined #openstack-infra | 18:22 | |
| *** vkozhukalov has joined #openstack-infra | 18:22 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/requirements: Update openstack-doc-tools to 0.4 https://review.openstack.org/69685 | 18:23 |
| *** gokrokve has quit IRC | 18:23 | |
| fungi | neat! that zuul thread on the gerrit list is a great opportunity for expanding the userbase | 18:24 |
| *** gokrokve_ has quit IRC | 18:24 | |
| fungi | who do we have planning to attend the gerrit summit? | 18:24 |
| *** vjay has quit IRC | 18:24 | |
| clarkb | I think jeblair and zaro submitted talks | 18:24 |
| reed | any suggestion for what is going wrong here? http://paste.openstack.org/show/62034/ | 18:25 |
| *** nati_ueno has joined #openstack-infra | 18:25 | |
| *** jp_at_hp has quit IRC | 18:25 | |
| zaro | i have submitted a general talk fro CI infra. but will probably not go indepth on zuul alone. | 18:26 |
| zaro | fungi: you wanna go? | 18:26 |
| *** gokrokv__ has quit IRC | 18:26 | |
| clarkb | reed: I think we need to see your template | 18:27 |
| mordred | neat! on zuul on gerrit list | 18:27 |
| clarkb | reed: I am guessing there is somethign in the template the substituter is falling over on | 18:28 |
| *** lcheng_ has quit IRC | 18:28 | |
| reed | clarkb, the settings.py file http://paste.openstack.org/show/62035/ | 18:29 |
| anteaya | I think markmcclain hit the bug that was mentioned about dealing with the new rax nodes: http://logs.openstack.org/53/69653/1/check/check-tempest-dsvm-neutron-pg-isolated/48bc4c6/console.html | 18:29 |
| anteaya | it looks to me like this was executed on one of the new rax nodes, was it not? | 18:29 |
| anteaya | do we have a bug number for that yet? | 18:29 |
| *** jpich has quit IRC | 18:30 | |
| *** VijayT has joined #openstack-infra | 18:30 | |
| fungi | reed: what's in line 11 column 51 of the list you're processing? | 18:30 |
| reed | ah | 18:30 |
| clarkb | reed: $600-off | 18:30 |
| mgagne | is there a way to abandon a change that's not mine? | 18:30 |
| fungi | mgagne: one of the gerrit admins can. what change number? | 18:31 |
| clarkb | reed: need to escape the $ I thinnk | 18:31 |
| reed | even inside the """ ? | 18:31 |
| reed | geez | 18:31 |
| clarkb | reed: $$600-off is what you need | 18:32 |
| clarkb | reed: yes because it is substituting $name and $code and so on, it things $600 needs substituting too | 18:32 |
| reed | gotcha | 18:32 |
| *** jooools has quit IRC | 18:33 | |
| fungi | aha, yes makes sense. that was line 11 of the EMAIL_TEMPLATE format string | 18:33 |
| *** DinaBelova_ is now known as DinaBelova | 18:33 | |
| reed | that was it, thanks clarkb | 18:33 |
| clarkb | anteaya: I am not familiar with that bug | 18:33 |
| clarkb | reed: np | 18:33 |
| anteaya | clarkb: fungi asked mordred about his request when he requested the rax nodes | 18:34 |
| fungi | zaro: if we already have two reps from os-infra going to the gerrit summit, i'm not sure it makes a ton of sense for me to fly to california in march for it | 18:34 |
| anteaya | mordred mentioned he requested 8G nodes | 18:35 |
| anteaya | did I mis-understand the conversation? | 18:35 |
| anteaya | that is entirely possible | 18:35 |
| harlowja | hi guys, could i get https://review.openstack.org/#/c/69534/ approved (followup from yesterdays changes) to make sure py2.6 works correctly | 18:35 |
| clarkb | anteaya: what is the bug though? | 18:35 |
| harlowja | *instead of currently failing (with not found) | 18:35 |
| fungi | anteaya: well, he requested enough resources (cpu et cetera) to accomodate a specific number of 8g performance flavor nodes | 18:35 |
| clarkb | harlowja: done | 18:36 |
| harlowja | thx clarkb | 18:36 |
| openstackgerrit | A change was merged to openstack-infra/config: Split config from projects list https://review.openstack.org/62187 | 18:36 |
| harlowja | u da man! | 18:36 |
| mordred | WOOT | 18:36 |
| anteaya | clarkb: you asked the question: 2014-01-28T18:08:32 | 18:36 |
| anteaya | ah okay then I did misunderstand | 18:36 |
| clarkb | by the way if no one has noticed I have been approving things with one +2 if I deem them low impact. I apologize in advance if any of my judgements are unsound | 18:37 |
| openstackgerrit | Peter Portante proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1266616 https://review.openstack.org/69670 | 18:37 |
| anteaya | so looking at this log file, ServerFault: Got server fault would that be a neutron failure or a host failure | 18:37 |
| fungi | clarkb: i've been doing that for weeks (not that you can tell from the remaining backlog) so i wholeheartedly endorse that method | 18:37 |
| anteaya | I don't know | 18:37 |
| fungi | anteaya: you have a log that's from, for greater context? | 18:38 |
| clarkb | anteaya: the line below the ServerFault tells you | 18:38 |
| anteaya | fungi: sorry: https://review.openstack.org/#/c/69653/ | 18:38 |
| zaro | fungi: i don't see any talks on the schedule from jeblair | 18:39 |
| fungi | anteaya: you probably already said and i just missed it. thanks | 18:39 |
| anteaya | ah a setupclass failure | 18:39 |
| clarkb | zaro: I may have been mistaken about jeblair submitting one then | 18:39 |
| anteaya | fungi: no I hadn't, I was jumping in in the middle again | 18:39 |
| *** krotscheck has joined #openstack-infra | 18:40 | |
| *** rossella_s has quit IRC | 18:41 | |
| *** che-arne has quit IRC | 18:42 | |
| harlowja | clarkb fungi another followup question, for these custom tox jobs, it appears the testr results don't showup, but only console.html, any idea | 18:45 |
| harlowja | example @ http://logs.openstack.org/05/69205/5/check/gate-taskflow-tox-py27-sa8-mysql/7dd7044/ | 18:45 |
| harlowja | maybe some other config missing? | 18:45 |
| notmyname | mordred: looks like had the same problem you (or bot) did https://review.openstack.org/#/c/69666/ | 18:45 |
| notmyname | mordred: works locally, git-review seems to be doing something odd | 18:45 |
| *** ewindisch is now known as zz_ewindisch | 18:46 | |
| fungi | harlowja: we probably need a separate publisher to collect them at the end of the job | 18:46 |
| clarkb | harlowja: the jobs use the other build script | 18:46 |
| clarkb | not the run testr script iirc | 18:46 |
| harlowja | hmmm | 18:46 |
| mordred | notmyname: git review -R right? (you definitely don't want it to rebase) | 18:46 |
| notmyname | mordred: ya | 18:46 |
| harlowja | fungi is there some special config option to make that happen? | 18:46 |
| fungi | harlowja: i'm looking | 18:47 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add murano with all subprojects https://review.openstack.org/69681 | 18:47 |
| harlowja | kk, thx fungi | 18:47 |
| *** jcoufal-mobile has joined #openstack-infra | 18:47 | |
| *** e0ne has quit IRC | 18:47 | |
| notmyname | mordred: ok, I don't have any more time to spend on it now. I'll try to find time tonight, if you haven't figured anything out before then | 18:48 |
| fungi | harlowja: the test-results publisher macro seems to grab '.testrepository/tmp*' and '**/*subunit_log.txt.gz' https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/macros.yaml#n297 | 18:49 |
| mordred | notmyname: wait - what are you trying to fix? | 18:49 |
| mordred | notmyname: it's possible that you and I may be talking about different things | 18:49 |
| clarkb | harlowja: fungi: only the run-testr script puts those files in the appropriate places | 18:51 |
| notmyname | mordred: I'm trying to get the 1.12.0 tag on master | 18:51 |
| mordred | notmyname: right. and there is a change up in gerrit for that - what's the problem? | 18:51 |
| *** sdague has quit IRC | 18:51 | |
| fungi | clarkb: harlowja: oh, right. so your test runner wrapper would need to do those same things itself | 18:51 |
| harlowja | hmmm | 18:51 |
| harlowja | k, let me double check where our testr is writing to | 18:52 |
| mordred | notmyname: the left-hand diff shows what the 'conflict' of merging the tag in would have been, the right hand shows the solution, which is to turn this into a merge which does not do anything | 18:52 |
| *** mrda_away has quit IRC | 18:52 | |
| notmyname | mordred: ah, ok. I guess my issue was not understanding what the left hand was showing then. ok. I'll approve/merge it | 18:53 |
| harlowja | fungi clarkb which is just i guess testr, maybe a special command neeeded @ https://github.com/stackforge/taskflow/blob/master/tox.ini#L51 | 18:53 |
| *** smarcet has left #openstack-infra | 18:53 | |
| notmyname | mordred: seems like it's removing some lines that aren't in master, so that's what's weird | 18:53 |
| *** alexpilotti has joined #openstack-infra | 18:54 | |
| *** e0ne has joined #openstack-infra | 18:55 | |
| *** sdague has joined #openstack-infra | 18:56 | |
| *** mrda_away has joined #openstack-infra | 18:58 | |
| harlowja | fungi thinking that i can have that tox.ini just output the files in the right locations, and that should maybe just do it :-P | 19:00 |
| fungi | harlowja: yep, ought to | 19:00 |
| mordred | notmyname: yeah - the display of merge commits in gerrit is interesting - and since we never work with them, it's even weirder | 19:00 |
| fungi | meeting time! | 19:01 |
| *** pblaho has joined #openstack-infra | 19:02 | |
| openstackgerrit | A change was merged to openstack-infra/config: Ensure taskflow py26 environments are using centos6 node https://review.openstack.org/69534 | 19:03 |
| *** jcoufal-mobile has quit IRC | 19:03 | |
| *** rcleere has quit IRC | 19:05 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack/requirements: Update python-openstacksdk from the global-requirements file https://review.openstack.org/69701 | 19:05 |
| *** mrodden has quit IRC | 19:07 | |
| *** devkulkarni has joined #openstack-infra | 19:08 | |
| openstackgerrit | Peter Portante proposed a change to openstack-infra/elastic-recheck: Update fingerprint for bug 1270608 https://review.openstack.org/69702 | 19:10 |
| devkulkarni | Hi there. have a question about running zuul related tests. When I run tox, it doesn't seem to complete (at least it has not finished for a long time now for me). any pointers/suggestions as to what might be going on? | 19:10 |
| *** VijayT has quit IRC | 19:11 | |
| clarkb | devkulkarni: we are in our meeting right now, can answer in about 50 minutes | 19:11 |
| devkulkarni | clarkb: sounds good. thanks!! | 19:12 |
| *** mrodden has joined #openstack-infra | 19:12 | |
| *** alexpilotti has quit IRC | 19:13 | |
| *** rcleere has joined #openstack-infra | 19:13 | |
| *** thedodd has quit IRC | 19:14 | |
| ArxCruz | fungi: besides the single_user_node, is there something else to configure? right now the nodes are disconnected from my jenkins, but there's no vm's being deployed in nodepool | 19:14 |
| *** alexpilotti has joined #openstack-infra | 19:16 | |
| ArxCruz | fungi: also I'm getting a lot of erros like this one http://paste.openstack.org/show/62037/ | 19:16 |
| ArxCruz | which i have no idea why, because if I call the get_info manually, I'm able to retrieve the information | 19:17 |
| openstackgerrit | A change was merged to openstack-infra/config: Unlaunchpadify projects.yaml https://review.openstack.org/62189 | 19:17 |
| *** max_lobur is now known as max_lobur_afk | 19:17 | |
| anteaya | devkulkarni: running tox produces no output (or very little) and takes quite a while depending on which project's tests you are running and the amount of resources available | 19:18 |
| *** dolphm is now known as dolphm_afk | 19:18 | |
| anteaya | also if you are running the tests in parrallel or not | 19:18 |
| fungi | ArxCruz: are you testing get_info using the same credentials (is your jenkins configured to allow the correct account)? also, we're in our weekly meeting, so i can't easily look into it for another 40 minutes or so | 19:19 |
| ArxCruz | fungi: yes | 19:19 |
| ArxCruz | fungi: oh, okay, sorry, we can talk later :) | 19:19 |
| fungi | ArxCruz: have you checked the jenkins.log for corresponding errors? | 19:19 |
| ArxCruz | tomorrow np :) | 19:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Track direct-release projects in projects.yaml https://review.openstack.org/62190 | 19:19 |
| *** gokrokve has joined #openstack-infra | 19:20 | |
| *** alexpilotti has quit IRC | 19:21 | |
| *** dkliban is now known as dkliban_afk | 19:22 | |
| *** gokrokve has quit IRC | 19:24 | |
| *** hemna has joined #openstack-infra | 19:24 | |
| *** hashar has joined #openstack-infra | 19:25 | |
| hemna | do we have any example logstash configs | 19:25 |
| hemna | for pushing logs into logstash in a devstack env ? | 19:25 |
| *** VijayT has joined #openstack-infra | 19:28 | |
| fungi | hemna: we have a client which relies on a zeromq plugin in jenkins to retrieve logs from completed jobs off our log publication server and feed them into logstash | 19:28 |
| clarkb | and we have logstash filters | 19:28 |
| hemna | hmm ok. | 19:28 |
| clarkb | hemna: openstack-infra/config/modules/openstack_project/templates/logstash/indexer.conf.erv iirc | 19:29 |
| clarkb | *.erb | 19:29 |
| hemna | I was just trying to logstash my normal devstack env (development machine) | 19:29 |
| *** sandywalsh has quit IRC | 19:29 | |
| mikal | Have any of you guys seen weird unit test failures on zuul? I can't explain the failure on https://review.openstack.org/#/c/65885/ | 19:29 |
| *** dkliban_afk is now known as dkliban | 19:29 | |
| clarkb | mikal: that may be the thing that usually affects python2.6 | 19:30 |
| pleia2 | clarkb: post meeting, I could use a quick sanity check re: fedora images defined in nodepool | 19:30 |
| sdague | mikal: there is a zuul unit test race | 19:31 |
| clarkb | mikal: it appears to be a job timeout, see if it goes away with recheck? jeblair is aware of it and working it as of lca iirc | 19:31 |
| mikal | clarkb: as in I should recheck and see if that helps? | 19:31 |
| clarkb | pleia2: sure | 19:31 |
| mikal | Ta | 19:31 |
| sdague | mikal: yeh, I hit it on one of mine. I think lifeless had an idea on it | 19:31 |
| mikal | I've never seen it before, but apparently its a thing! | 19:32 |
| *** jerryz has quit IRC | 19:33 | |
| fungi | devkulkarni reported something similar just above | 19:34 |
| *** _ruhe is now known as ruhe | 19:34 | |
| *** pblaho has quit IRC | 19:34 | |
| *** rwsu has quit IRC | 19:36 | |
| *** hashar has quit IRC | 19:39 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Updated grunt proxy server to rewrite to WSGI in dev https://review.openstack.org/69529 | 19:40 |
| *** rwsu has joined #openstack-infra | 19:40 | |
| *** sandywalsh has joined #openstack-infra | 19:41 | |
| *** david-lyle has quit IRC | 19:42 | |
| *** beagles is now known as beagles_brb | 19:42 | |
| *** lcheng has joined #openstack-infra | 19:43 | |
| *** hashar has joined #openstack-infra | 19:45 | |
| devkulkarni | anteaya: so I am running zuul tests (via tox) on my setup (ubuntu 13.10, 1GB ram, single core). btw, how do I run these in parallel? | 19:45 |
| anteaya | devkulkarni: please paste your tox.ini file, paste.openstack.org is a good option | 19:46 |
| *** denis_makogon has joined #openstack-infra | 19:47 | |
| devkulkarni | anteaya: http://paste.openstack.org/show/62040/ | 19:48 |
| sdague | mikal: yeh, it is sufficiently rare and obtuse no one has gotten to the bottom of it yet | 19:48 |
| anteaya | devkulkarni: so in your tox.ini you can see testr is invoked | 19:50 |
| *** AlexF_ has quit IRC | 19:50 | |
| *** sandywalsh has quit IRC | 19:50 | |
| anteaya | testr allows for parrallel testing (if the project you are testing is set up for parallel testing) | 19:50 |
| *** dkliban is now known as dkliban_afk | 19:51 | |
| devkulkarni | anteaya: ah I see. | 19:51 |
| clarkb | it also by default requires more than a single core | 19:51 |
| lyxus | Hello you, I waas wondering how long does it get the service account created ? | 19:51 |
| devkulkarni | clarkb: I see. so if I use multi-core machine these tests should run faster. let me give it a try. | 19:52 |
| anteaya | devkulkarni: https://wiki.openstack.org/wiki/Testr | 19:52 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add missed climate repos https://review.openstack.org/69680 | 19:52 |
| krotscheck | Can I get a few more opinions on https://review.openstack.org/#/c/67729/ ? | 19:53 |
| devkulkarni | anteaya: thanks for the link. don't know whether zuul is setup for parallel testing though (I am new to zuul) | 19:53 |
| lyxus | ( Service Accounts for 3rd party testing) | 19:53 |
| anteaya | devkulkarni: fair enough, I have never run tests on zuul before so I don't know off the top of my head either | 19:54 |
| anteaya | mikal: can you run zuul tests in parallel? | 19:54 |
| *** lcheng has quit IRC | 19:54 | |
| clarkb | anteaya: they run in parallel | 19:54 |
| mikal | anteaya: turbo hipster does this now, as does openstack's zuul | 19:54 |
| devkulkarni | clarkb: thanks for confirming. | 19:54 |
| mikal | anteaya: however, we wait until the end to report results | 19:54 |
| mikal | anteaya: you can however report results as they appear if you want to | 19:55 |
| clarkb | krotscheck: after meeting and lunch and stuff | 19:55 |
| clarkb | krotscheck: I am still head down in reviews when not doing the meeting or eating | 19:55 |
| krotscheck | clarkb: Thanks. Also, for your patch this morning. | 19:55 |
| anteaya | devkulkarni: wait until the end until you report, if you are submitting comments to our gerrit | 19:56 |
| krotscheck | Wouldn't mind getting fungi's opinion either. | 19:56 |
| fungi | krotscheck: my opinions are a commodity it seems. i'll see if i have any spares between the sofa cushions | 19:56 |
| anteaya | devkulkarni: if this is for a 3rd party system, one comment per event and hold off on voting until your system is stable | 19:57 |
| krotscheck | fungi: If they're really a commodity, can I get one at walmart? | 19:57 |
| anteaya | clarkb: nice job chairing today, btw | 19:57 |
| fungi | krotscheck: i only distribute through the 7-eleven. keeps my hipster cred intact | 19:58 |
| anteaya | you still are, 2 minutes left | 19:58 |
| devkulkarni | anteaya: sure. although, I don't think I am at a stage to submit anything yet :D need to just run the tests. this is not a 3rd party system. I am trying to help kraman and coolsvap with some of the changes that they have made in Zuul for Solum's needs. | 20:00 |
| *** freyes has joined #openstack-infra | 20:00 | |
| fungi | anteaya: right, i read his questions as "running zuul's tests" not "using zuul to run tests" | 20:00 |
| krotscheck | fungi: I dunno, for hipster cred you're going to need to like something that has potential to be cool. I don't think 7-11 will ever be that. | 20:00 |
| SergeyLukjanov | lifeless, hi | 20:00 |
| lifeless | SergeyLukjanov: hi | 20:01 |
| fungi | krotscheck: proof i'm no hipster i guess | 20:01 |
| SergeyLukjanov | lifeless, are you already testing dib builded images in tripleo? | 20:01 |
| SergeyLukjanov | lifeless, or publishing them as artifacts? | 20:01 |
| krotscheck | fungi: Or maybe you're a meta-hipster | 20:01 |
| lifeless | SergeyLukjanov: we're not publishing them yet, they are local to the test run | 20:01 |
| *** jcoufal has quit IRC | 20:01 | |
| SergeyLukjanov | lifeless, ok | 20:01 |
| lifeless | we'd like to be able to publish and reuse and so on | 20:01 |
| clarkb | pleia2: give me a few minutes before we talk fedora | 20:01 |
| clarkb | will ping in a bit | 20:01 |
| fungi | krotscheck: i'm so hip i can't see over my own pelvis | 20:01 |
| *** melwitt has joined #openstack-infra | 20:01 | |
| *** Sukhdev has joined #openstack-infra | 20:02 | |
| krotscheck | fungi: You're so hip, you're titanium. | 20:02 |
| * fungi needs to take a quick break, and then jump on reviews | 20:02 | |
| fungi | lyxus: i'll try to catch up on open bugs and ml threads requesting gerrit service accounts sometime today | 20:02 |
| pleia2 | clarkb: ok | 20:02 |
| anteaya | devkulkarni: great, I hope you create great things | 20:02 |
| anteaya | fungi: yes, my warning trigger finger is a little nervous | 20:03 |
| *** vkozhukalov has quit IRC | 20:03 | |
| *** rfolco has quit IRC | 20:05 | |
| clarkb | anteaya: thanks | 20:06 |
| clarkb | pleia2: now good? | 20:06 |
| clarkb | I need to grab lunch and make dinner shortly | 20:06 |
| clarkb | but can talk fedora first | 20:06 |
| * clarkb is going to attempt to make adobo. I will probably fail but adobo is too good to not try | 20:07 | |
| * anteaya googles adobo | 20:07 | |
| clarkb | dims: my plan is to put your version of the zmq event plugin on jenkins-dev and check the events coming out of it then upgrade | 20:08 |
| clarkb | I guess I should do that before restart 02 and 03 for SCP plugin and I can do both plugins at once | 20:08 |
| dims | k | 20:08 |
| anteaya | looks worthy of the attempt | 20:08 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Don't blow up on an empty core team list https://review.openstack.org/69711 | 20:09 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add murano with all subprojects https://review.openstack.org/69681 | 20:09 |
| *** freyes has quit IRC | 20:09 | |
| *** rcleere has quit IRC | 20:09 | |
| *** sandywalsh has joined #openstack-infra | 20:09 | |
| pleia2 | clarkb: hey, yes | 20:11 |
| *** yolanda_ has quit IRC | 20:11 | |
| pleia2 | clarkb: so I just need a quick update on when the fedora issues happend and what the current status is, looking at the config files yesterday it looks like they are still defined? | 20:11 |
| *** ArxCruz has quit IRC | 20:12 | |
| clarkb | pleia2: yeah I believe most of the stuff is still in place, it wasn't an issue until we asked nodepool to build images | 20:12 |
| *** mrodden1 has joined #openstack-infra | 20:12 | |
| clarkb | pleia2: ArxCruz and dprince had done the work the first time around, let me see if I can find the commits | 20:12 |
| fungi | clarkb: did it cause noepool to blow up? i don't think it did | 20:13 |
| * krotscheck is lying in wait for any of the infra core reviewers to indicate that they're reviewing again. | 20:13 | |
| *** nicedice has joined #openstack-infra | 20:13 | |
| fungi | clarkb: it looked like nodepool | 20:13 |
| fungi | er | 20:13 |
| fungi | like nodepool was blocking on not getting responses from jenkins02 | 20:13 |
| clarkb | pleia2: ebd8cddd6e0d52ab681624fd09bdb7e7bb936e2e was the revert commit | 20:13 |
| *** mrodden has quit IRC | 20:13 | |
| clarkb | fungi: this was before the holidays | 20:13 |
| fungi | oh that | 20:14 |
| clarkb | pleia2: and has links to the things reverted | 20:14 |
| clarkb | pleia2: it is possible Fedora 20 doesn't have this problem | 20:14 |
| russellb | speaking of node types ... i put up another pass on the "precise with saucy kernel" patch, trying to stomp top gate bug | 20:14 |
| russellb | https://review.openstack.org/#/c/69445/ | 20:15 |
| pleia2 | clarkb: so late last week we added the fedora 20 tripleo thing, did that have an impact? | 20:15 |
| clarkb | pleia2: does that help? basically nodepool sshs into the VMs and sudos a bunch of stuff, but it uses paramiko which doesn't allocate a tty and sudo on fedora 19 didn't like that | 20:15 |
| clarkb | pleia2: not sure. Did it build images successfully? if so then I don't think it had the same impact | 20:15 |
| lifeless | fungi: https://review.openstack.org/#/c/69309/ would be particularly good to land | 20:15 |
| pleia2 | hmm | 20:16 |
| pleia2 | so no, only fedora images are being built in the tripleo cloud | 20:17 |
| pleia2 | err only precise | 20:17 |
| pleia2 | but I thought https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb#n212 would mean it would try to build precise too | 20:18 |
| pleia2 | unless nodepool hasn't been restarted | 20:18 |
| pleia2 | I keep saying the wrong thing :) "mean it would try to build FEDORA too" | 20:19 |
| clarkb | nodepool doesn't need to be restarted to get new config | 20:20 |
| clarkb | and yes I think line 212 would mean images should be built | 20:20 |
| clarkb | I can do a nodepool list after lunch to see what is going on | 20:20 |
| *** gokrokve has joined #openstack-infra | 20:20 | |
| pleia2 | I did a nova list on the tripleo cloud, only see precise | 20:20 |
| pleia2 | lifeless: is there something I'm missing here? | 20:21 |
| clarkb | krotscheck: so your tests will use selenium? | 20:21 |
| krotscheck | clarkb: Some of them, yes. | 20:22 |
| fungi | russellb: comments left on that one | 20:22 |
| krotscheck | clarkb: It's executed via grunt though, for easier orchestration. | 20:22 |
| fungi | nm. i see clarkb left the exact same comments basically | 20:22 |
| *** mfer has quit IRC | 20:23 | |
| lifeless | pleia2: context? | 20:23 |
| clarkb | krotscheck: and dist/** will contain documentation? | 20:23 |
| russellb | fungi: clarkb thanks guys :) | 20:23 |
| pleia2 | lifeless: so you made the changes last week to add fedora to nodepool, but the tripleo cloud doesn't seem to be building them | 20:23 |
| krotscheck | clarkb: dist/** contains the minified app. mordred thought it'd be a good idea to sanity-check upload it much like docs so we can proof it. | 20:23 |
| pleia2 | (or even trying, it won't actually work fully yet) | 20:23 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Don't blow up on an empty core team list https://review.openstack.org/69711 | 20:23 |
| clarkb | krotscheck: gotcha | 20:23 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add murano with all subprojects https://review.openstack.org/69681 | 20:24 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Add a new node type for precise with saucy kernel https://review.openstack.org/69445 | 20:24 |
| lifeless | pleia2: it tries and fails | 20:24 |
| pleia2 | lifeless: ok, do we have record somewhere of what is happening so I can tackle that? | 20:24 |
| lifeless | pleia2: nodepool log | 20:24 |
| pleia2 | lifeless: right now I'm just running devstack scripts manually on an F20 image | 20:24 |
| lifeless | pleia2: you can run nodepool against a real cloud like cd-overcloud :) | 20:25 |
| pleia2 | lifeless: ok, so if I tried this myself with nodepool and the fedora config I'll see what errors it's getting? | 20:25 |
| clarkb | lifeless: pleia2: my guess is it fails for the same reason | 20:25 |
| lifeless | pleia2: I know you know how to do that :) | 20:25 |
| clarkb | paramiko with no tty breaks sudo on fedora 20 | 20:25 |
| pleia2 | lifeless: right-o, thanks | 20:25 |
| pleia2 | clarkb: yeah, probably so | 20:25 |
| clarkb | nodepool needs to have paramiko allocate a tty | 20:25 |
| fungi | lifeless: clarkb: isn't 69272 going to be a problem because the only job invoked in the check pipeline is non-voting? | 20:25 |
| *** beagles_brb is now known as beagles | 20:25 | |
| *** gokrokve has quit IRC | 20:25 | |
| clarkb | is it insecure to run sudo without a tty? what is the logic behind that? | 20:25 |
| *** devkulkarni has quit IRC | 20:25 | |
| fungi | also, seems i can either read irc or review changes. wow there's a lot of conversation | 20:26 |
| lifeless | fungi: I guess we should keep noop ? | 20:26 |
| clarkb | fungi: lifeless: no I believe it works fine | 20:26 |
| clarkb | I think we inadverdently tested that and it worked | 20:26 |
| fungi | clarkb: okay. i'll look to see whether we have any precedent for it | 20:26 |
| pleia2 | ok lunching now, will be back on nodepool after | 20:28 |
| krotscheck | clarkb: Thanks for the review. I'm guessing jeblair's comment on the previous patchset is not enough to get this merged? | 20:28 |
| fungi | pleia2: if i get time, i can look through the image log and see where it's going bad | 20:29 |
| *** mrodden1 is now known as mrodden | 20:29 | |
| *** dkliban_afk is now known as dkliban | 20:29 | |
| openstackgerrit | Chmouel Boudjnah proposed a change to openstack-infra/config: Don't log openstack-gate anymore. https://review.openstack.org/69714 | 20:29 |
| clarkb | krotscheck: typically we need two +2s. I have been breaking that rule for simple changes that are low impact. This adds a bunch of new stuff so having more eyes would be nice | 20:29 |
| fungi | krotscheck: i'm looking over 67729 now | 20:30 |
| krotscheck | fungi: Thanks | 20:30 |
| openstackgerrit | lifeless proposed a change to openstack-infra/nodepool: Expose paramiko's get_pty parameter. https://review.openstack.org/69715 | 20:30 |
| lifeless | pleia2: clarkb: try this ^ | 20:30 |
| fungi | eek... i cringe when i see Xvfb mentioned in unattended test scripts. i will get over it though | 20:32 |
| *** whoops has quit IRC | 20:32 | |
| clarkb | fungi: :) it means we can run firefox on our machines and have tests interact with firefox. That isn't scary at all | 20:32 |
| *** devkulkarni has joined #openstack-infra | 20:33 | |
| fungi | um. right. of course not | 20:33 |
| fungi | that's why i drink | 20:33 |
| krotscheck | fungi: Just don't look too hard at what's going on there, or you'll realize its... *gasp*... javascript! | 20:33 |
| fungi | yeah, thinking too hard is not one of my failings, thankfully | 20:33 |
| *** e0ne has quit IRC | 20:34 | |
| *** dkliban has quit IRC | 20:34 | |
| *** malini_afk is now known as malini | 20:34 | |
| fungi | this pattern has been tested on a headless system already, presumably | 20:35 |
| fungi | and without sudo privs | 20:35 |
| krotscheck | fungi: That's how I run my tests. Having a browser constantly pop up during the selenium runs is obnoxious, xvfb makes it not annoy me. | 20:36 |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 20:37 | |
| *** lcestari has quit IRC | 20:37 | |
| *** gokrokve has joined #openstack-infra | 20:38 | |
| *** thuc has quit IRC | 20:40 | |
| *** whoops has joined #openstack-infra | 20:41 | |
| *** thuc has joined #openstack-infra | 20:41 | |
| fungi | krotscheck: so the only thing which i think is probably unnecessary is that you have copy-on-failure for the draft build which results from the unit tests run, even though your failure message is going to be the standard pointer to the console log | 20:42 |
| *** gokrokve has quit IRC | 20:43 | |
| fungi | if you want to keep the drafts from failed unit test runs, then cool. just be aware you have to construct the urls yourself | 20:43 |
| krotscheck | fungi: We fix that two commits down the line: https://review.openstack.org/#/c/67520/3/modules/openstack_project/files/jenkins_job_builder/config/javascript.yaml | 20:43 |
| fungi | (i assume they will be tiny and few enough as to not really contribute to disk utilization anyway) | 20:43 |
| fungi | oh, hah | 20:43 |
| fungi | wfm | 20:43 |
| fungi | this is what i get for reviewing changes one at a time ;) | 20:44 |
| *** pcrews has quit IRC | 20:44 | |
| *** johnthetubaguy has quit IRC | 20:44 | |
| krotscheck | fungi: Seriously. You should just load it all into your brain at once like clarkb. | 20:45 |
| fungi | i'd blow a gasket | 20:45 |
| *** thuc has quit IRC | 20:45 | |
| fungi | krotscheck: were you super eager to address clarkb's one non-necessary comment request on that change (or is it also addressed in a later patchset)? if not, approving | 20:46 |
| anteaya | fungi I'd imagine a disk space error | 20:46 |
| fungi | anteaya: nope. i'm purely analog | 20:46 |
| *** DinaBelova is now known as DinaBelova_ | 20:47 | |
| anteaya | more reliable | 20:47 |
| fungi | alcohol combustion engine powered | 20:47 |
| anteaya | distilled in a chevy rad? | 20:47 |
| fungi | and stockpiled in mason jars, yes | 20:47 |
| openstackgerrit | Philippe Godin proposed a change to openstack-infra/jenkins-job-builder: Add best and worst status details for conditional-step plugin. https://review.openstack.org/69608 | 20:47 |
| krotscheck | fungi: Not really, though I can throw that in there if it means I the review won't take 3 weeks this time. | 20:47 |
| fungi | krotscheck: nah, you're all set | 20:48 |
| pabelanger | Question about the jenkins slaves, are you using SSH public keys for them to communication with jenkins masters? Versus over HTTP / HTTPS? | 20:48 |
| krotscheck | fungi: Awesome, thanks! | 20:49 |
| fungi | pabelanger: we have the jenkins masters connect to the slaves via ssh | 20:49 |
| openstackgerrit | A change was merged to openstack-infra/config: Use nodeenv via tox to do javascript testing https://review.openstack.org/67729 | 20:50 |
| pabelanger | fungi, Oh, so, the master initiates the connect, not the slaves? | 20:50 |
| krotscheck | Cooooooool | 20:51 |
| krotscheck | Ok, so now there's this one: https://review.openstack.org/#/c/67731/ | 20:51 |
| fungi | krotscheck: yep. i always look at dependency chains when i review so i can keep my train of thought. already on it | 20:53 |
| krotscheck | fungi: Ok, I'll shut up. | 20:53 |
| fungi | pabelanger: yes, master ssh's to each slave and launches the slave agent on demand | 20:54 |
| *** ruhe is now known as _ruhe | 20:54 | |
| fungi | pabelanger: basically when jenkins is told to connect (or reconnect) to a slave it launches or restarts the slave agent there | 20:54 |
| *** markmc has quit IRC | 20:54 | |
| *** e0ne has joined #openstack-infra | 20:55 | |
| fungi | pabelanger: however, be aware that jenkins is designed to allow slaves to request running arbitrary things on the master via that channel, so don't just assume it's providing suitable slave isolation that way | 20:55 |
| *** UtahDave has joined #openstack-infra | 20:56 | |
| pabelanger | fungi, interesting. Any reason why we connect directly from the master, vs the slave connecting back? I think there is a plugin for SSH support | 20:56 |
| fungi | pabelanger: we isolate all our sensitive slaves on a separate master which doesn't run jobs containing arbitrary code to help mitigate that issue | 20:56 |
| fungi | pabelanger: not sure if it was assumed to be more secure (at least we don't have to install sensitive private keys on all our slaves, so there's that benefit) | 20:57 |
| pabelanger | roger | 20:58 |
| fungi | pabelanger: but since we have public ip addresses on all slaves currently, it's not any more convenient to have slaves connect to the master (to get around nat for example) | 20:58 |
| lyxus | fungi, awesome ! Thanks ! | 20:58 |
| *** gyee has quit IRC | 20:58 | |
| *** hashar has quit IRC | 21:00 | |
| *** _ruhe is now known as ruhe | 21:00 | |
| fungi | krotscheck: on 67731 it looks like it's still not removing the "copy-after-failure: true" from the dist publisher for that gate-{name}-js-unittests template. was there another change to address that, or do you see benefit to uploading docs-drafts artifacts for failed unit tests? (keeping in mind that you will have to construct the url for them on your own) | 21:02 |
| *** markwash has quit IRC | 21:02 | |
| krotscheck | fungi: Yeah, now that's one patch down the stream. | 21:03 |
| krotscheck | fungi: And no - there's no reason to upload the app on a failure. | 21:03 |
| fungi | i guess the other patch got disassociated from 67731 in that case (it has no children in gerrit) | 21:04 |
| *** talluri has quit IRC | 21:05 | |
| krotscheck | fungi: It's outdated, I'm in merge hell at the moment. Give me 10 | 21:06 |
| *** hashar has joined #openstack-infra | 21:06 | |
| fungi | krotscheck: no rush. i'm in review hell, so i understand ;) | 21:06 |
| fungi | i think those spheres of hell are mostly adjacent | 21:07 |
| * krotscheck says screwit and abandons the prvious patch | 21:07 | |
| krotscheck | Nuke from orbit, start over | 21:07 |
| openstackgerrit | A change was merged to openstack-infra/config: Run gate-triple-deploy on tripleo-ci changes. https://review.openstack.org/69272 | 21:08 |
| *** UtahDave has quit IRC | 21:10 | |
| openstackgerrit | A change was merged to openstack-infra/config: Run gate-tripleo-deploy on devstack-gate changes. https://review.openstack.org/69273 | 21:10 |
| openstackgerrit | A change was merged to openstack-infra/config: Use the openstack pypi mirror for tripleo images. https://review.openstack.org/69309 | 21:11 |
| *** yassine has joined #openstack-infra | 21:12 | |
| clarkb | krotscheck: :/ nuking from orbit removes old review context | 21:12 |
| fungi | krotscheck: stick with us for a few months and you'll be a bona fide git rebase -i ninja | 21:13 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Enable tempest/savanna gate tests https://review.openstack.org/68066 | 21:13 |
| krotscheck | clarkb: In this case, nobody's reviewed it. | 21:14 |
| clarkb | krotscheck: so about that | 21:14 |
| krotscheck | clarkb: Wait - I lied. | 21:14 |
| clarkb | when you have really long patch series (again I know mostly mordred's fault) you are not likely to get reviews on the 7th change when the first has -1s | 21:15 |
| clarkb | I think that is partially why some of these have languished | 21:15 |
| krotscheck | clarkb: In contrast, if a review languishes I'm going to continue tweaking and iterating on it until it makes someone happy. | 21:16 |
| anteaya | krotscheck: if you inclined, if you spend some time doing reviews, that helps the reviewers | 21:17 |
| krotscheck | anteaya: I do. Mostly on storyboard though. | 21:17 |
| clarkb | krotscheck: thats fair just pointing out that it helps to do two things. not have long patch series if possible (logically independent changes are great) and address comments in earlier changes | 21:17 |
| krotscheck | anteaya: I'll expand my scope. | 21:17 |
| anteaya | says the person whose review statts are currently dismal | 21:17 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Updated tarball/client dist upload for javascript https://review.openstack.org/69722 | 21:17 |
| anteaya | krotscheck: thanks | 21:18 |
| *** AlexF_ has joined #openstack-infra | 21:18 | |
| fungi | krotscheck: you'll get particularly adept at using rebase's edit feature to change patches in the middle of topic branches | 21:18 |
| fungi | it's actually quite slick once you get used to it | 21:19 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Updated tarball/client dist upload for javascript https://review.openstack.org/69722 | 21:19 |
| krotscheck | Alright, fungi: That patch addresses the copy-on-failure thing as well as clark's request for additional comments. | 21:20 |
| *** e0ne has quit IRC | 21:20 | |
| *** ArxCruz has joined #openstack-infra | 21:21 | |
| devkulkarni | clarkb, anteaya: fyi, I have been able to run zuul's tests successfully. on a 8 core 8 GB machine (12.10 ubuntu) it took about two and half minutes. had to install python-dev though before tox could run | 21:21 |
| fungi | but i agree with clarkb... unless the changes are complex enough to be separate commits yet interdependent enough to require one another, stacking changes is somewhat counterproductive to the review process (with one obvious exception being changes which are mostly baked but are likely to merge-conflict with one another) | 21:21 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Enable tempest/savanna gate tests https://review.openstack.org/68066 | 21:21 |
| anteaya | devkulkarni: congratulations | 21:21 |
| anteaya | yes installing python-dev makes sense | 21:21 |
| krotscheck | fungi: I'm inclined to agree as well. I just happened to inherit some of mordred's patches. | 21:21 |
| fungi | krotscheck: you are experiencing it first hand without causing it. lucky you | 21:22 |
| krotscheck | Ehn, he's just hazing the new guy | 21:22 |
| krotscheck | In the meantime I've gotten javascript into infra without making anyone's head explode (too much). I call that a win. | 21:23 |
| *** AlexF_ has quit IRC | 21:23 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 21:24 | |
| clarkb | lifeless: quick question, for that nodepool non blocking delete change. is that something you had planned on retesting? | 21:24 |
| clarkb | lifeless: also, thanks for the pty change. At some point I should get a nodepool setup against my hpcloud account to test stuff | 21:25 |
| *** dprince has quit IRC | 21:25 | |
| *** hashar has quit IRC | 21:25 | |
| lifeless | clarkb: its' pretty easy, see the nodepool README these days :) | 21:25 |
| lifeless | clarkb: no, I don't have time to retest nodepool atm | 21:26 |
| clarkb | lifeless: ok, I will try spinning up a testbed for it myself then thanks | 21:26 |
| pleia2 | lifeless: testing nodepool now, but I'm not getting very far because I immediately get piles of: OverLimit: Quota exceeded, too many key pairs. (HTTP 413) (Request-ID: req-2f64bc4b-84ab-4a7a-8bb4-848b81eba00f) | 21:27 |
| anteaya | shopping for upvotes to remove logging from openstack-gate: https://review.openstack.org/#/c/69714/ | 21:27 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Update colors, header https://review.openstack.org/69723 | 21:27 |
| lifeless | pleia2: are you using your account? | 21:28 |
| *** dolphm_afk is now known as dolphm | 21:28 | |
| lifeless | pleia2: I suggest testing wit yyour tripleo-cd credentials | 21:28 |
| *** rcleere has joined #openstack-infra | 21:28 | |
| fungi | pleia2: also, you can get novacient to list/delete keypairs on that provider | 21:28 |
| fungi | pleia2: we've found they have a tendency to accumulate and people don't realize there are quotas on them | 21:29 |
| pleia2 | lifeless: was using the infra one | 21:29 |
| pleia2 | fungi: ah :) | 21:29 |
| *** devkulkarni has left #openstack-infra | 21:30 | |
| jog0 | sdague: where is the new home of the gate job failure gprahite logs? | 21:30 |
| clarkb | pleia2: any chance you want to test lifeless' nodepool stack of changes after testing the get_pty change | 21:30 |
| fungi | pleia2: aborted server launches can often leave orphaned keypairs behind, and the pile up over time | 21:30 |
| pleia2 | clarkb: will do | 21:30 |
| fungi | they pile up | 21:30 |
| anteaya | krotscheck: thank you | 21:31 |
| jog0 | because I used those every day | 21:31 |
| lifeless | yah we should make just one keypair | 21:31 |
| lifeless | and use it | 21:31 |
| lifeless | rather than per-server. IMNSHO. | 21:32 |
| lifeless | maybe one per image or something. | 21:32 |
| krotscheck | anteaya: Seemed simple enough | 21:32 |
| SergeyLukjanov | clarkb, fungi, sdague, I've made an async gate jobs for savanna, could you, please, take a look on config update? https://review.openstack.org/#/c/68066 | 21:32 |
| anteaya | krotscheck: yes, some are | 21:32 |
| sdague | jog0: it currently is homeless | 21:33 |
| *** cody-somerville has quit IRC | 21:33 | |
| anteaya | for the simple ones, it then allows one core to drive by and approve if there are a bunch of +1s | 21:33 |
| jog0 | sdague: :( lets fix that | 21:34 |
| pleia2 | fungi: thanks, that sorted it | 21:34 |
| jog0 | can we put it back on status.o.o/e-r at the bottom for now | 21:34 |
| jog0 | until the new home is built | 21:34 |
| *** coolsvap has quit IRC | 21:35 | |
| *** JpMaxMan has quit IRC | 21:36 | |
| sdague | jog0: so you realize it's completely flawed right | 21:36 |
| sdague | ? | 21:36 |
| jog0 | sdague: compleatly flawed? how so? | 21:36 |
| sdague | because of the interupt exception | 21:36 |
| sdague | interupt == failed | 21:36 |
| sdague | which it isn't | 21:36 |
| sdague | so all the failed numbers are impacted by gate depth | 21:36 |
| sdague | which is why they were corolated so strongly before | 21:37 |
| jog0 | bug Bug 1270309 - jenkins java.lang.InterruptedException | 21:37 |
| sdague | and why the fail numbers seemed higher | 21:37 |
| jog0 | which has had 43 hits in last 24 hours | 21:37 |
| jog0 | sdague: that makes sense | 21:37 |
| jog0 | in that case we should get that bug fixed | 21:37 |
| jog0 | clarkb: ^ | 21:38 |
| jog0 | what is involved in fixing that bug | 21:38 |
| jog0 | sdague: I forgot about that bug :/ | 21:38 |
| fungi | jog0: feel like contributing upstream development on jenkins? | 21:38 |
| jog0 | fungi: thats where it is done? | 21:39 |
| fungi | it's full of javalicious fun times | 21:39 |
| sdague | jog0: https://review.openstack.org/#/c/65700/ - so jeblair rightly kind of objects to that being a dedicated page. Honestly, I'm kind of out of juice on that one. | 21:39 |
| sdague | I don't think it should be on the e-r page, because I think it's mostly a distraction | 21:39 |
| fungi | jog0: yeah, the underlying problem is that when jenkins receives an abort/cancel sometimes it shuts things down in such an order that it reports a failure for the running job | 21:39 |
| fungi | jog0: we've just assumed we'd live with it until we no longer use jenkins, i think | 21:40 |
| jog0 | fungi: yeah that sounds reasonable sadly | 21:40 |
| anteaya | jog0: I hear you though, I do like graphs | 21:41 |
| jog0 | anteaya sdague: I just want to know if there was a spike in gate failures or not | 21:41 |
| anteaya | but it helps if they reflect the data accurately | 21:41 |
| anteaya | right | 21:41 |
| jog0 | if anyone has an idea how to do that ... | 21:41 |
| anteaya | do the individual graphs give you enough of that? | 21:42 |
| jog0 | not really | 21:42 |
| *** zehicle has joined #openstack-infra | 21:42 | |
| anteaya | I scroll the top 5 graphs | 21:42 |
| *** eharney has quit IRC | 21:42 | |
| jog0 | well we do know the classification rate is still very high | 21:42 |
| sdague | jog0: use the ES data, then you can filter out the Interupt fails | 21:42 |
| jog0 | sdague: that doesn't deal with duplicates | 21:42 |
| sdague | it does if you code it | 21:42 |
| jog0 | oh you mean: total fail - bug 1270309 | 21:43 |
| sdague | I mean get all the build_uuids that fail on that | 21:43 |
| fungi | yeah, i think from a data mining perspective, finding a way to filter out failures resulting from a java interrupt exception is going to get you the cleanest numbers | 21:43 |
| sdague | and process all the rest of the fails | 21:43 |
| sdague | and subtract those | 21:43 |
| fungi | count them as equivalent to abort/cancel (which i assume you currently ignore) | 21:44 |
| jog0 | sdague: fungi: yeah | 21:44 |
| sdague | yeh, we're looking for that Finished: line | 21:44 |
| sdague | so those wouldn't have it | 21:44 |
| clarkb | fungi: yup properly aborted jobs are ignored | 21:45 |
| *** zehicle_at_dell has quit IRC | 21:45 | |
| * jog0 is looking into updating elastic-recheck-graph | 21:48 | |
| *** dkliban has joined #openstack-infra | 21:49 | |
| sdague | it's going to get ugly | 21:50 |
| *** zehicle has quit IRC | 21:51 | |
| sdague | jog0: honestly, I think a more useful thing to do would be to make a version of the graphs page which is *only* gate resets | 21:51 |
| jog0 | sdague: what does that tell us? | 21:51 |
| sdague | we want to know what to fix | 21:51 |
| sdague | what is killing us | 21:51 |
| jog0 | how does knowing just the number of gate resets help with that? | 21:52 |
| clarkb | AaronGr: left a comment on that change, think you should update the change to have cron or no cron | 21:52 |
| sdague | jog0: it would be the bug list with only the gate resets | 21:52 |
| sdague | that's what's actually killing us | 21:52 |
| sdague | right now those numbers are all queues | 21:52 |
| AaronGr | clarkb: no problem. i'll re-submit with that change. | 21:53 |
| *** dizquierdo has joined #openstack-infra | 21:54 | |
| jog0 | sdague: you mean have an option to only list gate queue failures on status.o.o/e-r | 21:54 |
| *** packet has joined #openstack-infra | 21:57 | |
| clarkb | fungi: any changes you have seen that you +2'd that could use a second +2? | 21:58 |
| *** rcleere has quit IRC | 21:58 | |
| clarkb | fungi: AaronGr: https://review.openstack.org/#/c/68771/ just adds the ability to diable a user but doesn't actually disable them correct? | 21:58 |
| sdague | jog0: yeh | 21:58 |
| clarkb | fungi: AaronGr: eg I don't have to worry too much about approving that | 21:58 |
| fungi | clarkb: not sure. i need to take a break to cook dinner here in a bit... https://review.openstack.org/#/q/watchedby:fungi@yuggoth.org+-owner:fungi@yuggoth.org+status:open+-status:workinprogress+-status:draft+-Approved%3E=1+CodeReview=2+-CodeReview%3C=-1+-Verified-1,n,z | 21:58 |
| fungi | oh, i guess the watchedby is probably wrong for you | 21:59 |
| AaronGr | fungi: correct. just adds a function to let you disable users, non-impacting. | 21:59 |
| *** zz_ewindisch is now known as ewindisch | 21:59 | |
| clarkb | AaronGr: ok I will approve | 21:59 |
| *** SergeyLukjanov is now known as SergeyLukjanov_ | 21:59 | |
| fungi | clarkb: yeah, would have been nice to have a few weeks ago ;) | 22:00 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Open port 8000 for heat-api-cfn for public neutron net https://review.openstack.org/69276 | 22:00 |
| stevebaker | clarkb: let me know what you think of the rule in ^ | 22:01 |
| *** Sukhdev has quit IRC | 22:01 | |
| openstackgerrit | A change was merged to openstack-infra/config: Extend user module, add 'disable user' https://review.openstack.org/68771 | 22:02 |
| clarkb | stevebaker: looks great | 22:02 |
| clarkb | stevebaker: I can deal with rebasing with the other change depending on how they merge | 22:03 |
| stevebaker | clarkb: ok. what kind of propagation delay happens after it lands? | 22:03 |
| clarkb | stevebaker: good question and it isn't great. Will need an image rebuild cycle which typically happens once a day but can be babysitted by hand if we want to go quicker | 22:04 |
| jog0 | sdague: POC coming shortly | 22:04 |
| clarkb | stevebaker: fungi, stevebaker's change above is anotehr good one to review if you have time :) | 22:04 |
| stevebaker | hmm, the checks failed | 22:05 |
| fungi | clarkb: yeah, saw it. i'm good approving it if you want to rebase 69442 | 22:05 |
| fungi | once it's clean | 22:05 |
| clarkb | fungi: fine with me | 22:06 |
| fungi | (rocketships) | 22:06 |
| clarkb | silly rocketships | 22:06 |
| clarkb | stevebaker: ^ | 22:06 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Open port 8000 for heat-api-cfn for public neutron net https://review.openstack.org/69276 | 22:06 |
| openstackgerrit | Aaron Greengrass proposed a change to openstack-infra/config: Remove hardcoded config assumptions, cleanup variables https://review.openstack.org/66072 | 22:07 |
| stevebaker | thats better, the check passes | 22:09 |
| * fungi steps away for a bit to make with the food | 22:10 | |
| *** harlowja is now known as harlowja_away | 22:10 | |
| *** sarob has joined #openstack-infra | 22:11 | |
| clarkb | by the way logstash kibana users. searching all time is bad juju | 22:11 |
| fungi | don't anger the tiki gods | 22:11 |
| *** eharney has joined #openstack-infra | 22:12 | |
| clarkb | anteaya: can you abandon https://review.openstack.org/#/c/69458/2 if it isn't needed | 22:12 |
| krotscheck | fungi: So, keep-hierarchy: false? | 22:12 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Make files in reviewday docroot accessible https://review.openstack.org/69734 | 22:13 |
| *** amotoki has quit IRC | 22:14 | |
| fungi | krotscheck: oh, good point, that may be the default, in which case you're fine | 22:14 |
| *** gyee has joined #openstack-infra | 22:14 | |
| openstackgerrit | A change was merged to openstack-infra/config: Open port 8000 for heat-api-cfn for public neutron net https://review.openstack.org/69276 | 22:14 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Make files in reviewday docroot accessible https://review.openstack.org/69734 | 22:14 |
| fungi | clarkb: do you know what it defaults to in the scp publisher? | 22:15 |
| clarkb | http://ci.openstack.org/jenkins-job-builder/publishers.html#publishers.scp says false | 22:15 |
| krotscheck | fungi: It is the default, but it helps to be explicit. Will update. | 22:16 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Updated tarball/client dist upload for javascript https://review.openstack.org/69722 | 22:16 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Only plot gate failures in status.o.o/elastic-search https://review.openstack.org/69735 | 22:16 |
| jog0 | sdague: ^ that just switches to gate failures only on status.o.o/e-r | 22:17 |
| fungi | clarkb: thanks. i should rtfm | 22:17 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Only plot gate failures in status.o.o/elastic-recheck/ https://review.openstack.org/69735 | 22:17 |
| wenlock_ | hey, im hoping a review can be done on this one : https://review.openstack.org/#/c/51425/ i have some time for updates if needed | 22:18 |
| sdague | jog0: so I don't think we want to switch | 22:18 |
| *** harlowja_away is now known as harlowja | 22:18 | |
| jog0 | sdague: I tend to agree | 22:18 |
| sdague | we need both, because gate volume is currently low | 22:18 |
| jog0 | we want to show both | 22:18 |
| jog0 | sdague: so to do so requires some UI magic that I don't have | 22:18 |
| sdague | so now would be the time for subpages | 22:18 |
| jog0 | I think | 22:18 |
| sdague | jog0: just make them different pages with our own nav embedded | 22:18 |
| sdague | the uncat page could come in as well | 22:19 |
| jog0 | sdague: you want to take a first pass at the own nav? | 22:19 |
| anteaya | clarkb: as of yesterday I felt it was needed, I haven't had time to go back and do my reading to bring it up with jog0 again | 22:19 |
| jog0 | my web UI is horrible | 22:19 |
| *** jcoufal has joined #openstack-infra | 22:19 | |
| jog0 | web UI foo | 22:19 |
| *** thuc has joined #openstack-infra | 22:19 | |
| sdague | I'm actually kind of buried at the moment | 22:19 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added no_api env https://review.openstack.org/68610 | 22:19 |
| jog0 | sdague: heh kk aren't we all | 22:20 |
| clarkb | anteaya: I don't think it is needed based on jog0's comment | 22:20 |
| * anteaya tries to find the url for the unclassified failures again | 22:20 | |
| *** thuc has quit IRC | 22:20 | |
| stevebaker | clarkb: OK, the change has landed. Any chance you could ping me when the rebuild cycle is done? | 22:20 |
| anteaya | clarkb: I disagree with jog0 | 22:20 |
| * jog0 is right here | 22:20 | |
| clarkb | anteaya: whyfor? | 22:20 |
| jog0 | anteaya: ? | 22:20 |
| anteaya | and then had to get onto a plane before he and I could discuss it | 22:20 |
| *** thuc has joined #openstack-infra | 22:20 | |
| clarkb | stevebaker: sure, I will try to kick those off when I have a chance to sit with a terminal to nodepool open | 22:20 |
| stevebaker | \o/ | 22:21 |
| anteaya | because the fingerprint was not picking up unclassified failures with the same message | 22:21 |
| anteaya | as of yesterday | 22:21 |
| anteaya | jog0: do you have the unclassified url again? | 22:21 |
| clarkb | anteaya: if you look at the results in the logstash query you link they come in pairs as jog0 describes | 22:21 |
| anteaya | it would save me time trying to find it | 22:21 |
| jog0 | use status.openstack.org/elastic-recheck/ to find it | 22:21 |
| jog0 | ohh you men http://status.openstack.org/elastic-recheck/data/uncategorized.html | 22:22 |
| anteaya | clarkb: but there were failures that were unclassified with the exact same message as of yesterday | 22:22 |
| jog0 | mean* | 22:22 |
| *** SumitNaiksatam_ has joined #openstack-infra | 22:22 | |
| anteaya | jog0: yes thanks | 22:22 |
| *** SumitNaiksatam has quit IRC | 22:22 | |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 22:22 | |
| clarkb | anteaya: do you have an example of that? | 22:22 |
| clarkb | want to make sure there isn't a larger underlying problem | 22:22 |
| anteaya | clarkb: I am looking | 22:23 |
| jog0 | sdague: so when running the gate only patch and ignoring CI | 22:24 |
| jog0 | sdague: Bug 1273386 is the top with 3 fails in 24hrs | 22:24 |
| jog0 | top open bug | 22:24 |
| jog0 | so woot | 22:24 |
| sdague | cool | 22:25 |
| jog0 | sdague: so if we can get someone to create the nav bar I can do the rest | 22:26 |
| StevenK | Huh, someone fixed git-review | 22:28 |
| *** jcoufal has quit IRC | 22:28 | |
| clarkb | krotscheck: your comment on https://review.openstack.org/#/c/69204/3/modules/openstack_project/files/jenkins_job_builder/config/storyboard.yaml can you help me understand that? isn't the storyboard docs job for building sphinx docs and not related to javascript? | 22:28 |
| anteaya | clarkb: jog0 http://logs.openstack.org/71/65371/2/gate/gate-tempest-dsvm-neutron/27ab52f/console.html | 22:28 |
| anteaya | first failure under gate-tempest-dsvm-neutron : 6 Uncategorized Fails. 97.6% Classification Rate (246 Total Fails) | 22:29 |
| anteaya | http://status.openstack.org/elastic-recheck/data/uncategorized.html | 22:29 |
| *** AlexF_ has joined #openstack-infra | 22:29 | |
| anteaya | that looks like the same message to me | 22:29 |
| jog0 | krotscheck: ping, have some UI work | 22:29 |
| anteaya | but it is still on the unclassified page | 22:29 |
| *** zehicle_at_dell has joined #openstack-infra | 22:30 | |
| krotscheck | clarkb: In the tox patches we moved all the storyboard jobs from storyboard.yaml into templates in javascript.yaml. I was thinking that would also be something we want to do for docs, but then I realized that ruhe's talking about the API, which is a python thing. | 22:31 |
| krotscheck | jog0: What's up? | 22:31 |
| jog0 | anteaya: looking | 22:31 |
| *** e0ne has joined #openstack-infra | 22:31 | |
| anteaya | jog0: thanks | 22:31 |
| jog0 | http://logs.openstack.org/71/65371/2/gate/gate-tempest-dsvm-neutron/27ab52f/logs/screen-n-api.txt.gz#_2014-01-25_19_02_05_279 | 22:32 |
| jog0 | its in n-api at info level. clarkb do we collect that? | 22:32 |
| anteaya | I am prepared to be wrong, but if so, I need direction about how to write the query to classify that failure | 22:32 |
| clarkb | jog0: we should I am digging into it | 22:32 |
| jog0 | clarkb: thanks | 22:33 |
| krotscheck | clarkb: I just had a build fail looking for the nodeenv package - I'm guessing this is because it's not on our pypi mirror? If so, is that in openstack/requirements? | 22:33 |
| jog0 | anteaya: ^ thats your answer | 22:33 |
| jog0 | krotscheck: so we have this page http://status.openstack.org/elastic-recheck/ | 22:33 |
| *** AlexF_ has quit IRC | 22:33 | |
| anteaya | jog0: logstash doesn't collect n-api at the info level? | 22:33 |
| anteaya | so what is the next step to get that failure classified? | 22:34 |
| jog0 | right now it queries all queues (check, gate, etc) - but we want an option to only list hits on the gate queue | 22:34 |
| jog0 | krotscheck: I have a patch to do the elasticSearch changes https://review.openstack.org/#/c/69735/ | 22:34 |
| jog0 | krotscheck: but not sure what best way to update UI | 22:34 |
| ruhe | krotscheck: clarkb: i'll add a comment that we agreed in irc to keep storyboard-doc template in storyboard.yaml | 22:34 |
| anteaya | e-r queries all queues? (trying to ensure I have the correct vision of what "it" is referencing) | 22:34 |
| jog0 | either subpages with a nav bar, or a button or something | 22:34 |
| jog0 | anteaya: see clarkb's comment | 22:35 |
| clarkb | build_uuid:"27ab52ffbfec4f32a1ab23ef0361360a" AND filename:"logs/screen-n-api.txt" it didn't get the file | 22:35 |
| jog0 | clarkb: ahh | 22:35 |
| jog0 | anteaya: ^ we hit a different bug | 22:35 |
| krotscheck | jog0: Ok, gimme a few moments to decipher what you mean by all that :) | 22:35 |
| clarkb | so for one reason or another it didn't index that file (logstash doesn't try to hard to work around derpage because there is so much going o | 22:35 |
| clarkb | jog0: anteaya: so I don't think we should update the query | 22:36 |
| anteaya | jog0: so this log is referencing a different bug with a similar message | 22:36 |
| jog0 | krotscheck: http://status.openstack.org/elastic-recheck/ has a bunch of graphs my POC https://review.openstack.org/#/c/69735/ changes the data that is graphed | 22:36 |
| clarkb | it won't fix the problem | 22:36 |
| *** e0ne has quit IRC | 22:36 | |
| anteaya | okay I can abandon this patch and offer a new patch with a brand new query | 22:36 |
| jog0 | sdague: ^ perhaps you have a better vision of what the UI should be | 22:36 |
| clarkb | anteaya: no, it is the same bug from e-r perspective. But the reason it didn't match was we didn't index the file | 22:36 |
| *** mriedem has quit IRC | 22:36 | |
| clarkb | anteaya: a new query won't help | 22:36 |
| anteaya | clarkb: ah okay, standing by | 22:36 |
| clarkb | anteaya: we need to make logstash fail less | 22:36 |
| anteaya | clarkb: ah | 22:36 |
| *** dkranz has quit IRC | 22:36 | |
| anteaya | clarkb: so yes to abandon the patch? | 22:37 |
| jog0 | anteaya: yeah | 22:37 |
| anteaya | k | 22:37 |
| clarkb | anteaya: the problem here is we are indexing a ton of data. If some of it falls on the floor we say oh well and move on otherwise we would back up and bad things happen | 22:37 |
| krotscheck | jog0: I'm guessing you're looking for an incremental change? | 22:38 |
| clarkb | krotscheck: can you link to the failure? and no you shouldn't be using the mirror | 22:38 |
| clarkb | at least not only the mirror | 22:38 |
| jog0 | krotscheck: yes | 22:38 |
| krotscheck | clarkb: http://logs.openstack.org/10/68610/4/check/gate-storyboard-webclient-js-unittests/f9709ce/console.html | 22:38 |
| anteaya | clarkb: right | 22:38 |
| anteaya | abandoned | 22:39 |
| krotscheck | jog0: Tall order, let me sketch a few things out - 10 mins or so. | 22:39 |
| jog0 | krotscheck: doing sub pages with a nav bar may be the easiest | 22:39 |
| jog0 | where we can include status.openstack.org/elastic-recheck/data/uncategorized.html as well | 22:40 |
| jog0 | krotscheck: thanks! | 22:40 |
| krotscheck | jog0: That's where I'm leaning. | 22:40 |
| jog0 | krotscheck: awesome | 22:40 |
| clarkb | krotscheck: so you want to know something funny | 22:40 |
| clarkb | krotscheck: you will like this | 22:40 |
| *** CaptTofu has quit IRC | 22:41 | |
| krotscheck | clarkb: ..... | 22:41 |
| krotscheck | clarkb: ..... that sounds like a trap. | 22:41 |
| clarkb | krotscheck: I think what happened htere is you used the mirror because run-jsbuild.sh does not run the select mirror function which means you are stuck with whatever the previous jobs mirror selection was | 22:41 |
| clarkb | krotscheck: to fix that add http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/run-tox.sh#n32 to run-jsbuild.sh and I can push it through | 22:41 |
| *** CaptTofu has joined #openstack-infra | 22:41 | |
| krotscheck | clarkb: Ok, let me help out jog0 first. | 22:42 |
| *** stevemar has joined #openstack-infra | 22:44 | |
| *** jnoller_ has quit IRC | 22:45 | |
| *** CaptTofu has quit IRC | 22:46 | |
| fungi | hooray for cross-project pollination/cooperation! as harry tuttle said, "we're all in it together" | 22:48 |
| ruhe | clarkb: one question about https://review.openstack.org/#/c/69204 . it expects "ci/storyboard" to exist on 173.203.107.207. i believe this dir should be created manually? | 22:49 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Run the tests for the python-openstacksdk project https://review.openstack.org/69667 | 22:49 |
| stevemar | lookin for some help debugging an error i'm seeing in my jenkins results, any takers? | 22:50 |
| *** yamahata has joined #openstack-infra | 22:50 | |
| clarkb | ruhe: the scp plugin will create it if necessary | 22:51 |
| ruhe | clarkb: that's handy | 22:51 |
| fungi | ruhe: it's handy until you want part of the path to that to be a symlink | 22:52 |
| *** mattoliverau has quit IRC | 22:52 | |
| lifeless | fungi: can I get a +A on https://review.openstack.org/#/c/69309/ ? | 22:52 |
| fungi | at which point it gets removed and recreated | 22:52 |
| clarkb | dhellmann: what is the project that clients use that isn't prettytable but does output of stuff, cliff? | 22:52 |
| dhellmann | cliff uses prettytable | 22:53 |
| dhellmann | it just wraps it differently so clients aren't using it directly | 22:53 |
| fungi | lifeless: did you mean something else? that's approved/merged already | 22:53 |
| clarkb | dhellmann: thanks | 22:53 |
| *** dizquierdo has quit IRC | 22:53 | |
| anteaya | 6 +1s and a +2 from fungi for removing logging from openstack-gate: https://review.openstack.org/#/c/69714/ | 22:53 |
| lifeless | fungi: hah, gocool | 22:54 |
| lifeless | stale browser page | 22:54 |
| anteaya | ready for a +A anytime | 22:54 |
| lifeless | fungi: thanks! | 22:54 |
| fungi | anteaya: done. smells like consensus from the denizens to me | 22:55 |
| clarkb | hopefully a meeting doesn't start in 5 minutes :) | 22:55 |
| anteaya | agreed | 22:55 |
| anteaya | ha ha ha | 22:55 |
| *** thuc has quit IRC | 22:55 | |
| *** prad has quit IRC | 22:55 | |
| fungi | eep, right | 22:55 |
| *** thuc has joined #openstack-infra | 22:56 | |
| openstackgerrit | A change was merged to openstack-infra/config: Don't log openstack-gate anymore. https://review.openstack.org/69714 | 22:57 |
| fungi | i'm starting to think we should get rid of the notify/subscribe on that config and just schedule a weekly reload midnight utc saturday night/sunday morning until someone gives it a graceful option | 22:57 |
| clarkb | fungi: or just manually kick it like manage-projects :/ I don't actually like that idea | 23:00 |
| *** thuc has quit IRC | 23:00 | |
| krotscheck | jog0: http://bootply.com/109172 as a "First blush" | 23:01 |
| *** melwitt1 has joined #openstack-infra | 23:01 | |
| *** mattoliverau has joined #openstack-infra | 23:01 | |
| clarkb | cyeoh: comment on https://review.openstack.org/#/c/69542/ | 23:02 |
| krotscheck | jog0: Does that show for you? | 23:02 |
| * krotscheck will have to tweak the typography a bit of course... | 23:02 | |
| clarkb | I see a cat in the render. I assume it worked :) | 23:03 |
| *** melwitt has quit IRC | 23:03 | |
| cyeoh | clarkb: thx - will fix it up - I think it should be possible to find this in n-cpu.txt... | 23:03 |
| krotscheck | clarkb: Yes, well, that's placekitten.com :) | 23:03 |
| jog0 | krotscheck: awesome | 23:03 |
| *** pcrews has joined #openstack-infra | 23:03 | |
| mattoliverau | Morning everyone. | 23:03 |
| *** openstack has joined #openstack-infra | 23:04 | |
| -wolfe.freenode.net- [freenode-info] channel flooding and no channel staff around to help? Please check with freenode support: http://freenode.net/faq.shtml#gettinghelp | 23:04 | |
| jog0 | krotscheck: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/web/share/index.html | 23:04 |
| jog0 | sdague: ^^ | 23:04 |
| krotscheck | anteaya: Thanks! | 23:04 |
| mattoliverau | anteaya: how are you going this fine day! | 23:04 |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 23:04 |
| *** obondarev has quit IRC | 23:04 | |
| anteaya | mattoliverau: I have been absoring salt info, and am full atm | 23:04 |
| clarkb | mattoliverau: is jhesketh about? | 23:04 |
| anteaya | sitting in a tutorial at salfconf | 23:04 |
| krotscheck | jog0: Is that a "Hey do this for me" or a "can you review this?" :) | 23:04 |
| anteaya | mattoliverau: and thou? | 23:04 |
| anteaya | mattoliverau: how go the boxes? | 23:05 |
| jog0 | krotscheck: this is a, hey I don't know how to do this halp | 23:05 |
| clarkb | mattoliverau: I was hoping he would have an opinion on whether or not his zuul patches should come with tests | 23:05 |
| krotscheck | jog0: Righto | 23:05 |
| jog0 | krotscheck: if you do the layout I can do the content | 23:05 |
| *** obondarev has joined #openstack-infra | 23:05 | |
| mattoliverau | clarkb: I'm in Sydney, jhesketh is currently on a plane to join us.. so he should be online at some point, but not for a few hours. | 23:06 |
| jog0 | sdague: we need more kittens http://bootply.com/109172 | 23:06 |
| krotscheck | jog0: Seems fair. I've got a few storyboard patches I need to get through first. | 23:06 |
| jog0 | krotscheck: no problem, thanks a ton | 23:06 |
| clarkb | mattoliverau: gotcha thnaks. no rush | 23:06 |
| mattoliverau | anteaya: good, in Sydney for the next few days.. so I have escaped the boxes for a while :) | 23:07 |
| anteaya | mattoliverau: I'm sure your wife loves that you have left her with the boxes | 23:07 |
| anteaya | happy sydney | 23:07 |
| anteaya | any cooler there? | 23:07 |
| sdague | krotscheck: yeh, right now it's basically inheriting style from status.o.o.... so you'll probably want to cross check with jeblair on outer frame changes | 23:08 |
| mattoliverau | anteaya: yeah, I bet she does :P Yeah, only 28 today, nice and cool! | 23:08 |
| anteaya | woohoo | 23:08 |
| anteaya | avoid the 30 | 23:08 |
| clarkb | krotscheck: sdague: oh right, it tries to be consistent | 23:08 |
| anteaya | 's | 23:08 |
| anteaya | mattoliverau: be sure to take a gift home, but don't pack it in a box | 23:08 |
| mattoliverau | anteaya: lol! great idea. | 23:09 |
| anteaya | :D | 23:09 |
| clarkb | pleia2: any luck with get_pty testing? | 23:09 |
| krotscheck | sdague: I'll probably custom write the CSS for the tabs to get jog0 on his way. | 23:09 |
| fungi | clarkb: the status page style is eventually consistent | 23:10 |
| krotscheck | sdague: I usually just use bootply as a quick sandbox. | 23:10 |
| sdague | krotscheck: that would be totally cool | 23:10 |
| pleia2 | clarkb: I'm pretty sure it worked since I'm getting a new error now, but it's been tricky to debug | 23:10 |
| *** ruhe is now known as _ruhe | 23:10 | |
| clarkb | pleia2: new errors are the best | 23:11 |
| * fungi will take a moment to look at the current fedora issue as it stands in production, and make sure that's what's wrong | 23:11 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 23:11 |
| *** mrda_away is now known as mrda | 23:11 | |
| openstackgerrit | Julien Danjou proposed a change to openstack-infra/pypi-mirror: Whitelist posix-ipc https://review.openstack.org/69743 | 23:12 |
| pleia2 | fungi: will need to do some key cleanup again, they keep piling up and not helping the situation :) | 23:12 |
| * clarkb is sad that the number of changes merged today is far less than yesterday, but ESOMANYREVIEWS | 23:12 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Added select_mirror to javascript build. https://review.openstack.org/69744 | 23:13 |
| *** oubiwann_ has quit IRC | 23:13 | |
| fungi | clarkb: yesterday were the low-hanging fruit. now we're down to mostly dense change material | 23:13 |
| krotscheck | clarkb: ^^ | 23:13 |
| fungi | also, i was way too busy with administrivia today. hopefully tomorrow i can pull my weight on that | 23:14 |
| anteaya | is jeblair still sick? | 23:15 |
| fungi | krotscheck: lgtm | 23:15 |
| anteaya | I know mordred is in a car or on a plane or something | 23:15 |
| krotscheck | I think jeblair's on a plane to FOSDEM | 23:15 |
| anteaya | ah | 23:15 |
| anteaya | good, I am glad he is feeling better then | 23:15 |
| fungi | anteaya: he's somwhere over the mariana trench | 23:16 |
| anteaya | hopefully high in the air then | 23:16 |
| fungi | no, wait, the atlantic rift | 23:16 |
| fungi | forgot he was going through dc | 23:16 |
| krotscheck | jog0: Ok, how do you want these changes. layered onto your own patch? | 23:16 |
| anteaya | cool | 23:16 |
| clarkb | krotscheck: question about https://review.openstack.org/#/c/69722/3/modules/openstack_project/files/jenkins_job_builder/config/javascript.yaml why are we copying dist/** to logs/LOG_PATH? | 23:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Added select_mirror to javascript build. https://review.openstack.org/69744 | 23:16 |
| anteaya | some water filled abyss in any case | 23:17 |
| clarkb | krotscheck: isn't all of that content available as a tarball on the tarballs site? | 23:17 |
| jog0 | krotscheck: sure | 23:17 |
| krotscheck | clarkb: Mostly as a sanity check to catch situations like "Hey, this tarball upload didn't work, did we build the right thing?" | 23:19 |
| clarkb | krotscheck: but in either case you will have a tarball and a list of files that match each other right? | 23:19 |
| clarkb | I guess I don't see the value in copying the data twice | 23:20 |
| fungi | i concur with clarkb, we're not in the business of testing whether 'tar -czf' works as advertised | 23:20 |
| krotscheck | clarkb: Fair point. I dunno, we're currently dealing with a situation where the post gate tarball was never uploaded. | 23:20 |
| *** dcramer_ has quit IRC | 23:21 | |
| clarkb | krotscheck: that won't be fixed or debugged by that change. | 23:21 |
| clarkb | at least not by that part of it | 23:21 |
| fungi | krotscheck: that should result in an error in the console og | 23:21 |
| fungi | log | 23:21 |
| clarkb | I updated my review | 23:22 |
| fungi | krotscheck: where/when did it fail? | 23:22 |
| fungi | i'm happy to help troubleshoot | 23:22 |
| krotscheck | fungi: Actually, it never ran. | 23:23 |
| *** alexpilotti has joined #openstack-infra | 23:23 | |
| fungi | ahh, that's an important detail... under what circumstances were you expecting it to run? | 23:23 |
| krotscheck | fungi: And that was a couple of patches ago, so we wanted to land the build updates first before diving in on why the trigger never executed. | 23:23 |
| fungi | in the post pipeline? | 23:23 |
| krotscheck | fungi: Yup | 23:23 |
| krotscheck | fungi: Let me amend that. It ran once. Then never again. | 23:23 |
| fungi | i should be able to hunt the log from the last change which merged to that repo | 23:24 |
| *** dims has quit IRC | 23:24 | |
| krotscheck | afk- making changes in accordance to clark's comments | 23:25 |
| *** julim has quit IRC | 23:27 | |
| fungi | also, public service announcement, most everyone in this channel probably has a summit invite as of the past few hours. if you're coming, please sign up asap. this year there's a cutoff where if you don't sign up in the first few weeks you're technically supposed to come up with the difference between the early-bird pricing and normal pricing (so please don't procrastinate) | 23:27 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/config: Updated tarball/client dist upload for javascript https://review.openstack.org/69722 | 23:27 |
| clarkb | fungi: we ready to approve https://review.openstack.org/#/c/69445/6 ? | 23:28 |
| clarkb | fungi: I think we have until march before that becomes a problem | 23:28 |
| clarkb | but yes, sign up | 23:28 |
| * clarkb does that | 23:28 | |
| fungi | clarkb: right, march is just around the corner ;) | 23:29 |
| fungi | krotscheck: http://logs.openstack.org/9e/9e4411efab99438c21d6fd02cb0cae32bb3f7579/post/storyboard-webclient-release-branch/4c48dae/ | 23:29 |
| clarkb | fungi: I need to step out in a bit for about half an hour. If I +2 69445 would you be able to babysit? | 23:30 |
| *** thuc has joined #openstack-infra | 23:30 | |
| fungi | clarkb: yes, for sure | 23:30 |
| *** thuc has quit IRC | 23:30 | |
| fungi | i'll be here all night (until i pass out anyway) | 23:30 |
| clarkb | fungi: :P approve at will | 23:31 |
| *** thuc has joined #openstack-infra | 23:31 | |
| fungi | clarkb: do you need/would you want me to rebuild nodepool devstack-precise images, or should we just wait for the timer? | 23:31 |
| clarkb | fungi: oh I suppose if you are in there go ahead | 23:32 |
| *** thuc has quit IRC | 23:32 | |
| fungi | i can do that | 23:32 |
| *** thuc has joined #openstack-infra | 23:32 | |
| clarkb | I was intending on doing it when I got around to testing zmq plugin on jenkins-dev | 23:32 |
| clarkb | great thanks | 23:32 |
| *** denis_makogon has quit IRC | 23:32 | |
| *** melwitt1 is now known as melwitt | 23:32 | |
| fungi | hpcloud-az2 does a poor job of successfully building images. the most recent devstack-precise image there is almost 4 days old there now | 23:34 |
| clarkb | :( | 23:34 |
| fungi | the last time i looked at the logs for it, we were hitting ssh timeouts after launch before puppeting | 23:35 |
| *** dims has joined #openstack-infra | 23:39 | |
| *** stevemar has quit IRC | 23:40 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add a new node type for precise with saucy kernel https://review.openstack.org/69445 | 23:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Genericize javascript release artifact creation https://review.openstack.org/67731 | 23:44 |
| *** oubiwann_ has joined #openstack-infra | 23:45 | |
| anteaya | so now that russellb's patch is merged, can we run a check experimental job on that node type? | 23:46 |
| *** paul-- has joined #openstack-infra | 23:46 | |
| *** pcrews has quit IRC | 23:46 | |
| anteaya | its all there in the commit message | 23:47 |
| * anteaya goes off to check experiemental on some neutron jobs | 23:47 | |
| anteaya | or should I wait for this patch to propegate? | 23:47 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Add support to run nova-api-metadata as separate binary https://review.openstack.org/68304 | 23:48 |
| russellb | anteaya: before the jobs can run nodepool has to build the image for this node type ... i'm not sure how often it's configured to rebuild them | 23:48 |
| *** sarob has quit IRC | 23:48 | |
| *** sarob has joined #openstack-infra | 23:49 | |
| anteaya | ah | 23:49 |
| anteaya | every 24 hours, not sure of the utc time | 23:49 |
| anteaya | unless it is kicked | 23:49 |
| russellb | OK | 23:49 |
| anteaya | I will wait until tomorrow then | 23:49 |
| anteaya | but thanks russellb, very happy to have this change in | 23:49 |
| russellb | np | 23:50 |
| russellb | not sure if it'll help or not, but it's worth trying to see what happens | 23:51 |
| russellb | and i always have fun learning more infra stuff :) | 23:51 |
| anteaya | agreed | 23:52 |
| anteaya | yes, regardless of the outcome I am keen to see the logs | 23:52 |
| anteaya | and yes, it is fun to learn more infra | 23:52 |
| *** sarob has quit IRC | 23:53 | |
| clarkb | anteaya: russellb: fungi indicated he would kick it | 23:53 |
| russellb | neat | 23:54 |
| clarkb | so possibly in another hour or so we can trigger those experimental jobs | 23:54 |
| clarkb | I need to test dims zmq plugin change now | 23:54 |
| anteaya | clarkb: ah cool, yes I saw the babysitting/kick conversation but didnt connect the dots | 23:54 |
| *** whoops has quit IRC | 23:54 | |
| anteaya | I lose | 23:54 |
| anteaya | woohoo | 23:54 |
| clarkb | that moment when you 'G' and expect firefox to go to end of page. I should reinstall vimperator | 23:55 |
| fungi | clarkb: anteaya: russellb: nodepool builds images on demand, so it should already be trying to build them as long as it thinks it needs nodes of tat type | 23:56 |
| russellb | oh, cool. | 23:56 |
| anteaya | I'll trigger some experimental jobs | 23:56 |
| clarkb | I am going to restart jenkins-dev nowish, you have been warned :) | 23:56 |
| anteaya | might as well see what happens | 23:56 |
| russellb | anteaya: great | 23:56 |
| fungi | clarkb: anteaya: russellb: and that's the problem--forgot to add nodes to a provider using that image :( | 23:57 |
| fungi | patch coming | 23:57 |
| anteaya | okay | 23:58 |
| anteaya | only hit check experimental on one so far | 23:59 |
| anteaya | if a patch is in check, commenting "check experimental" will not interfere with check tests executing properly, correct? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!