Monday, 2013-12-23

*** sthaha has joined #openstack-qa00:08
openstackrecheckopenstack/nova change: https://review.openstack.org/58191 failed tempest because of: https://bugs.launchpad.net/bugs/125489000:34
jamielennoxanyone with some grenade knowledge here? I can't see what's wrong here: https://launchpad.net/bugs/1262934 and it's preventing me landing reviews in keystone01:23
morganfainbergjamielennox, ick, granade issues?01:31
*** pmathews has joined #openstack-qa01:32
jamielennoxmorganfainberg: it's been the same thing for a week or so now, every now and then i get on and try to bug someone about it and nothing01:32
morganfainbergjamielennox, If you haven't gotten it resolved after christmas / holiday time, I'll follow up w/ folks for you during the day (US)01:33
jamielennoxreally starting to annoy me, because i have no idea how to even go about finding where it's coming from, i'm currently seeing if i can take the localrc from a grenade run and run the same thing locally01:33
morganfainbergjamielennox, see if we can rally some help01:33
morganfainbergjamielennox, yeah, :( wish i was more help on that front01:33
jamielennoxbut it appears like the branch setup is done by some sort of gate proxy and not grenade itself so i can't even figure out how to replicate it01:33
morganfainbergjamielennox, a lot of that work (iirc) is done in the jenkins slave scripts01:34
morganfainbergat least for most gate jobs01:34
jamielennoxyea, well the keystone_branch is master so somewhere they are pointing the repo at jenkins - but not in a way that i can use it again later01:34
morganfainberg=/01:34
jamielennoxmorganfainberg: it's the reason that there are no passing kds builds :( i need to get some reviews on that if we want to land it for I201:35
morganfainbergjamielennox, well, post holidays when we have more infra types back, i'll help you hunt someone down if you haven't already solved it01:36
morganfainbergi just don't know if there is much else i can do until people are done holiday-ing01:36
jamielennoxmorganfainberg: yea, i know - i've got the same thing, i won't be back till ike the 6th - was going for a last minute fix :)01:37
morganfainberghonestly, i should learn more about granade, but i have a couple patchsets i need to get up and some other things that need attention while holiday is in effect01:37
openstackgerritSean Dague proposed a change to openstack/tempest: rename old config object to TempestConfigPrivate  https://review.openstack.org/6352901:40
openstackgerritSean Dague proposed a change to openstack/tempest: remove last errant file parse warning  https://review.openstack.org/6364901:40
openstackgerritSean Dague proposed a change to openstack/tempest: remove unneeded __init__ file  https://review.openstack.org/6353001:40
openstackgerritSean Dague proposed a change to openstack/tempest: wip: experimental skip decorators  https://review.openstack.org/6354001:40
sdaguejamielennox: so how far have you debugged the issue? like where does it fail in the console, and what service is failing when that's the case? We keep pretty extensive logs for a grenade job, so you should have most of what you need to get to root cause there.01:48
jamielennoxsdague: hey! so i think that the old environment is being kept around and when we upgrade keystone we are getting a collision of the services that are still running01:53
jamielennoxsdague: https://review.openstack.org/#/c/63246/ is my test case because all it does is update the requirements to add WSME to keystone01:54
morganfainbergsdague, isn't it a sunday and holiday season?  (not that i'm complaining that you're around to help jamielennox out ;) )01:54
sdaguemorganfainberg: it is... so I'll have to run from the computer quickly01:54
morganfainbergsdague, well.. happy holidays man :)  have a good one.  (i'll let you back to chatting w/ jamielennox  now)01:54
jamielennoxsdague: i've been going through devstack following the install_stackforge stuff but that doesn't appear to be run from grenade01:55
sdaguejamielennox: if you read the console log you can see we are applying the new global requirements, which should have picked that up01:55
jamielennoxbut one problem is it's failing in swift-proxy but i can't see anywhere that swift-proxy actually has a dependency on wsme01:55
sdaguehttp://logs.openstack.org/46/63246/1/check/check-grenade-dsvm/23e94a6/console.html.gz#_2013-12-20_01_15_35_76901:55
*** harlowja has joined #openstack-qa01:55
*** pmathews has quit IRC01:56
sdaguedoes ceilometer have swift hooks?01:59
jamielennoxsdague: i've no idea - but i previously identified that the old version of ceilometer had the same conflicting requirements02:00
sdaguethe old wsme comes from ceilometer dragging it in - http://logs.openstack.org/46/63246/1/check/check-grenade-dsvm/23e94a6/console.html.gz#_2013-12-20_01_07_20_13702:00
sdaguethe reason swift is exploding is actually because wsme is upgraded02:00
sdagueand something, probably an entry point, requires the old version02:00
jamielennoxor just that ceilometer is still running with the older reqs when swift is being upgraded02:01
sdagueit's not that it's running02:01
sdagueswift is actually failing to start02:01
sdagueso something in it's load path requires the old version02:01
jamielennoxyep, but nothing i can see in swift in devstack is out of the swift tree other than authtoken02:02
jamielennoxunless grenade is setting that up differently02:02
sdagueso ceilometer works by injecting code into other projects02:02
sdaguethat's probably the issue02:02
sdagueand it probably comes down to the fact that ceilometer is installed on the old side, but we don't seem to be upgrading it.02:03
sdaguesearch for /opt/stack/old/ceilometer and /opt/stack/new/ceilometer02:04
openstackrecheckopenstack/neutron change: https://review.openstack.org/61168 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1256106 and https://bugs.launchpad.net/bugs/124906502:04
jamielennoxhuh, no new/ceilometer - so it's not being upgraded at all02:04
sdagueso probably the fastest path is to stop installing ceilometer on the old side until they have an upgrade script in there02:04
sdagueyeh02:04
jamielennoxor at least until after swift is trying to use it02:04
sdagueright02:05
sdagueor create an upgrade-ceilometer piece in grenade, and hope all it needs is setup_devel02:06
jamielennoxyea, nothing in grenade that would appear to do much for ceilometer, so it needs to not be not in enabled_services for gate jobs02:06
jamielennoxit would have to be ordered though right - ceilometer would have to be pretty much the first piece to be upgraded02:07
sdaguehttps://github.com/openstack-dev/grenade/blob/master/grenade.sh#L25602:07
sdagueso I expect we're taking the global service list in the grenade jobs (and after the big config / layout refactor something might have gotten tweaked)02:08
sdagueI thought we had the smaller service list here, I'm actually surprised the ceilometer was being added02:08
sdaguegiven all of it's issues it's probably better to pull it here02:08
sdaguehttps://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L49 is the logic that probably needs to be fixed02:10
sdagueanyway, I need to stop working now. I will be on tomorrow.02:10
sdaguethat's my last day for the year02:10
jamielennoxsdague: ok, i'll have a look - i don't yet see ceilometer mentioned their in infra02:11
sdaguethe issue you'll run into of devstack or grenade locally vs. devstack-gate, is the wrapper to bring up the d-g environment02:12
sdaguehttps://github.com/openstack-infra/devstack-gate/blob/master/README.rst - "Simulating Devstack Gate Tests"02:12
sdaguegood reading02:12
jamielennoxsdague: ah, that will be useful02:13
sdagueok, night folks02:13
jamielennoxthat was where i was stuggling to get passed debugging as the localrc in grenade won't actually help you test the change02:13
jamielennoxsdague: thanks for your help02:13
sdagueno prob02:13
openstackrecheckopenstack/tempest change: https://review.openstack.org/63540 failed tempest with an unrecognized error02:32
*** sthaha has quit IRC02:34
*** sthaha has joined #openstack-qa02:34
openstackrecheckopenstack/glance change: https://review.openstack.org/61595 failed tempest because of: https://bugs.launchpad.net/bugs/125489002:37
openstackrecheckopenstack/tempest change: https://review.openstack.org/63649 failed tempest because of: https://bugs.launchpad.net/bugs/125389602:46
openstackgerritJamie Lennox proposed a change to openstack-infra/devstack-gate: Remove ceilometer from the grenade gate  https://review.openstack.org/6366402:49
*** pmathews has joined #openstack-qa02:49
openstackrecheckopenstack/python-heatclient change: https://review.openstack.org/63661 failed tempest because of: https://bugs.launchpad.net/bugs/125389602:51
*** harlowja is now known as harlowja_away02:55
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543503:00
openstackrecheckopenstack/neutron change: https://review.openstack.org/60699 failed tempest because of: https://bugs.launchpad.net/bugs/125610603:06
openstackgerritMasayuki Igawa proposed a change to openstack/tempest: Add version test for Ceilometer cli  https://review.openstack.org/6362103:22
openstackgerritMasayuki Igawa proposed a change to openstack/tempest: Add version test for Ceilometer cli  https://review.openstack.org/6362103:23
openstackrecheckopenstack/nova change: https://review.openstack.org/58191 failed tempest because of: https://bugs.launchpad.net/bugs/125489003:30
*** psharma has joined #openstack-qa03:41
openstackrecheckopenstack/python-ceilometerclient change: https://review.openstack.org/63279 failed tempest because of: https://bugs.launchpad.net/bugs/124906503:45
openstackgerritSean Dague proposed a change to openstack/tempest: wip: experimental skip decorators  https://review.openstack.org/6354003:57
openstackrecheckopenstack/neutron change: https://review.openstack.org/63100 failed tempest because of: https://bugs.launchpad.net/bugs/125489004:08
openstackrecheckopenstack/python-novaclient change: https://review.openstack.org/63567 failed tempest because of: https://bugs.launchpad.net/bugs/125389604:11
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest with an unrecognized error04:20
*** HenryG has quit IRC04:22
openstackrecheckopenstack/neutron change: https://review.openstack.org/61351 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543504:25
openstackrecheckopenstack/python-heatclient change: https://review.openstack.org/63665 failed tempest because of: https://bugs.launchpad.net/bugs/125389604:28
*** harlowja_away is now known as harlowja04:31
*** harlowja has quit IRC04:35
openstackrecheckopenstack/tempest change: https://review.openstack.org/63649 failed tempest because of: https://bugs.launchpad.net/bugs/125389604:40
*** HenryG has joined #openstack-qa04:51
*** dkorn has quit IRC04:58
*** dkorn has joined #openstack-qa05:15
*** prekarat has joined #openstack-qa05:26
*** yfried has quit IRC05:29
openstackrecheckopenstack/nova change: https://review.openstack.org/61679 failed tempest because of: https://bugs.launchpad.net/bugs/125389605:31
openstackrecheckopenstack/heat change: https://review.openstack.org/63669 failed tempest with an unrecognized error05:39
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/1235435 and https://bugs.launchpad.net/bugs/125610605:39
openstackrecheckopenstack/neutron change: https://review.openstack.org/63047 failed tempest because of: https://bugs.launchpad.net/bugs/125489006:07
*** dkorn has quit IRC06:14
*** yfried has joined #openstack-qa06:19
openstackrecheckopenstack/heat change: https://review.openstack.org/63669 failed tempest because of: https://bugs.launchpad.net/bugs/125389606:25
*** macjack has joined #openstack-qa06:34
openstackrecheckopenstack/neutron change: https://review.openstack.org/63047 failed tempest because of: https://bugs.launchpad.net/bugs/125489006:53
*** yfried has quit IRC06:56
*** yfried has joined #openstack-qa07:02
*** yfried has quit IRC07:05
*** yfried has joined #openstack-qa07:06
*** dkorn has joined #openstack-qa07:07
openstackrecheckopenstack/neutron change: https://review.openstack.org/61351 failed tempest because of: https://bugs.launchpad.net/bugs/125489007:11
openstackrecheckopenstack/keystone change: https://review.openstack.org/63671 failed tempest because of: https://bugs.launchpad.net/bugs/125389607:16
openstackrecheckopenstack/neutron change: https://review.openstack.org/63056 failed tempest with an unrecognized error07:19
openstackrecheckopenstack/nova change: https://review.openstack.org/63670 failed tempest because of: https://bugs.launchpad.net/bugs/125389607:22
openstackrecheckopenstack/keystone change: https://review.openstack.org/63672 failed tempest because of: https://bugs.launchpad.net/bugs/1240256 and https://bugs.launchpad.net/bugs/125389607:22
openstackrecheckopenstack/glance change: https://review.openstack.org/63666 failed tempest with an unrecognized error07:30
openstackrecheckopenstack/cinder change: https://review.openstack.org/63533 failed tempest because of: https://bugs.launchpad.net/bugs/125389607:35
*** jlibosva has joined #openstack-qa07:39
*** prekarat has quit IRC07:45
yfriedenikanorov__: ping07:45
yfriedmarun: ping07:45
marunyfried: pong07:45
yfriedmarun: do you how many networks should be created for a test when using isolated-tenants?07:46
yfriedmarun: I'm seeing 2 extra nets even before the test creates its own network07:47
marunuh07:47
marunyfried: 2 extra nets owned by who?07:48
yfriedmarun: It seems like it's creating a network for the test's tenant07:48
marunyfried: that's kind of the point, yeah07:48
marunyfried: i'm not sure I understand what you see as a problem07:48
yfriedso why would the test need to create a new network?07:48
marunyfried: what kind of network?07:49
*** prekarat has joined #openstack-qa07:49
yfriedmarun: network_basic, and cross_tenant, both create networks for their own use07:49
yfriedmarun: "cross" creates a network for each tenant07:50
marunyfried: i'm afraid I still don't understand what you're asking me07:50
yfriedmarun: bear with me please07:50
marunyfried: presumably creating an isolated tenant creates a private network for that tenant07:50
yfriedmarun: so "cross" is trying to hold 5 networks (I'm taking public into account)07:51
marunyfried: that would be more for the nova tests07:51
marun'hold'?07:51
yfriedmarun: during "cross" execution - 5 networks should co-exist07:51
*** jamielennox is now known as jamielennox|away07:52
yfriedmarun: I'm having 3 issues:07:52
yfriedmarun: 1. you have public->tenant_network->test_created_network for each tenant. That's different than the public->test_created_network you had in mind designing the test. Could be an issue for some scenarios07:54
yfriedmarun: 2. what about quotas for more complicated tests?07:54
marunyfried: ah, ok07:55
yfriedmarun: 3. I'm using VLANs on my local setup and the test fails because I simply do not have enough VLANs available07:55
marunyfried: really?  how many vlans are you testing with?07:55
marunyfried: 2. quotas are per-tenant07:56
yfriedmarun: 3 - 1 for public and 2 extras07:56
marun1. i'd recommend detecting when you're using isolated and reusing the tenant network rather than creating a new one07:56
marunyfried: the limit is 4096 vlans, i'm not sure how you could possibly be running out07:56
yfriedmarun: 1. that's what I had in mind. I will open a bug for it07:56
maruncool07:57
yfriedmarun: VLANs are assigned to me by the office network manager07:57
yfriedmarun: how do I become a tempest member on launchpad? where do I register?07:58
marunyfried: apply to join here https://launchpad.net/~tempest-drivers07:58
openstackrecheckopenstack/nova change: https://review.openstack.org/59655 failed tempest with an unrecognized error08:03
marunyfried: regarding vlans, why are you using a real network?08:03
marunyfried: virtualization of hosts and networks is highly recommended for testing openstack.08:03
yfriedmarun: that's an internal issue - using multiple node setup, our hosts communicate using private vlans. This might not be the most efficient setup, but that's how it works08:05
yfriedmarun: we are working on "virtualizing" our testing setup08:05
yfriedmarun: new issue:08:06
yfriedmarun: I need help with configuring the gate for my external connectivity patch.08:07
yfriedmarun: I posted on the mailing list, but no response yet.08:07
*** sthaha has quit IRC08:08
marunyfried: how can I help?  I'm not sure external connectivity is even allowed at present.08:08
yfriedmarun: can you help? I'm being constantly bumped between infra, neutron, and qa rooms08:08
*** pmathews has quit IRC08:08
yfriedmarun: do we want this patch in? https://review.openstack.org/#/c/55146/ if so - it needs some kind of network to test agains08:09
yfriedt08:09
yfriedmarun: I was thinking using the gate's local dns08:10
marunyfried: that's not a bad idea.08:10
yfriedmarun: great. now, how do I get it done?08:12
marunyfried: I'm afraid I'm not the person to ask.  Regulars in this channel and #openstack-infra are your best bet.08:13
yfriedI'm not getting any response so far08:13
marunyfried: well, keep pinging them.  The tz difference is probably working against you in this regard.08:14
marunyfried: actually...08:14
yfriedmarun: I've been trying this for a few days now. Mailing list also...08:15
*** akamyshnikova_ has joined #openstack-qa08:15
marunyfried: It's the holidays for much of NA and Europe, so it's going to be slow going for the next week or two08:15
marunyfried: the people in NA that are most likely to be able to help are dkranz and mlavelle08:16
yfriedyfried: ok. I have a meeting with dkranz later today on this issue.08:16
yfriedmarun: is mlavelle redhat ?08:16
marunyfried: no, but he's championing tempest testing for neutron08:17
yfriedmarun: great. I'll talk to him as well08:18
*** prekarat has quit IRC08:22
*** prekarat has joined #openstack-qa08:23
*** kashyap has joined #openstack-qa08:27
openstackrecheckopenstack/neutron change: https://review.openstack.org/61351 failed tempest with an unrecognized error08:29
*** prekarat1 has joined #openstack-qa08:42
*** prekarat has quit IRC08:45
*** jlibosva has quit IRC08:50
*** jlibosva has joined #openstack-qa08:51
*** salv-orlando_ has joined #openstack-qa08:54
*** rossella_s has joined #openstack-qa08:56
openstackrecheckopenstack/neutron change: https://review.openstack.org/63047 failed tempest because of: https://bugs.launchpad.net/bugs/125144808:57
*** yassine has joined #openstack-qa08:57
*** salv-orlando has quit IRC08:57
*** salv-orlando_ is now known as salv-orlando08:57
*** safchain has joined #openstack-qa08:58
*** SergeyLukjanov has joined #openstack-qa09:02
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543509:03
*** vponomaryov has joined #openstack-qa09:07
*** salv-orlando has quit IRC09:07
*** salv-orlando has joined #openstack-qa09:07
*** prekarat has joined #openstack-qa09:13
*** prekarat1 has quit IRC09:15
*** prekarat1 has joined #openstack-qa09:19
*** prekarat has quit IRC09:22
openstackrecheckopenstack/neutron change: https://review.openstack.org/61351 failed tempest because of: https://bugs.launchpad.net/bugs/125489009:23
openstackrecheckopenstack/neutron change: https://review.openstack.org/61168 failed tempest because of: https://bugs.launchpad.net/bugs/125489009:25
*** shardy_afk is now known as shardy09:32
openstackrecheckopenstack/python-keystoneclient change: https://review.openstack.org/63408 failed tempest because of: https://bugs.launchpad.net/bugs/125389609:35
openstackrecheckopenstack/neutron change: https://review.openstack.org/63492 failed tempest because of: https://bugs.launchpad.net/bugs/125489009:39
*** tkammer has joined #openstack-qa09:45
openstackrecheckopenstack/requirements change: https://review.openstack.org/62028 failed tempest with an unrecognized error09:48
openstackrecheckopenstack/neutron change: https://review.openstack.org/59787 failed tempest because of: https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/1244255 and https://bugs.launchpad.net/bugs/125764110:00
*** wanghao has quit IRC10:02
*** wanghao has joined #openstack-qa10:02
openstackrecheckopenstack/neutron change: https://review.openstack.org/61067 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/1235435 and https://bugs.launchpad.net/bugs/125389610:02
*** SergeyLukjanov has quit IRC10:03
*** SergeyLukjanov has joined #openstack-qa10:04
openstackrecheckopenstack/heat change: https://review.openstack.org/62440 failed tempest with an unrecognized error10:04
*** zul has quit IRC10:06
*** zul has joined #openstack-qa10:08
*** tkammer has quit IRC10:09
*** tkammer has joined #openstack-qa10:18
*** ruhe has joined #openstack-qa10:18
openstackrecheckopenstack/neutron change: https://review.openstack.org/63585 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543510:21
*** rossella_s has quit IRC10:28
*** prekarat1 has quit IRC10:31
*** ruhe is now known as ruhe_10:42
*** yassine has quit IRC10:45
*** ruhe_ is now known as ruhe10:46
openstackrecheckopenstack/neutron change: https://review.openstack.org/52930 failed tempest because of: https://bugs.launchpad.net/bugs/125489010:46
*** rossella_s has joined #openstack-qa10:50
openstackrecheckopenstack/tempest change: https://review.openstack.org/63529 failed tempest because of: https://bugs.launchpad.net/bugs/125389610:58
openstackgerritValeriy Ponomaryov proposed a change to openstack/tempest: Add pluggable functionality for api and cli tests  https://review.openstack.org/6242911:00
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Commit test for neutron full gate tests  https://review.openstack.org/6371311:01
openstackrecheckopenstack/glance change: https://review.openstack.org/63666 failed tempest with an unrecognized error11:13
openstackrecheckopenstack-dev/devstack change: https://review.openstack.org/63641 failed tempest with an unrecognized error11:15
openstackrecheckopenstack/cinder change: https://review.openstack.org/63601 failed tempest with an unrecognized error11:16
openstackrecheckopenstack/neutron change: https://review.openstack.org/63585 failed tempest because of: https://bugs.launchpad.net/bugs/125610611:18
openstackrecheckopenstack/neutron change: https://review.openstack.org/63240 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1244255 and https://bugs.launchpad.net/bugs/125764111:26
openstackrecheckopenstack/neutron change: https://review.openstack.org/61168 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/1235435 and https://bugs.launchpad.net/bugs/125389611:26
*** rossella_s has quit IRC11:29
*** wanghao has quit IRC11:36
*** krtaylor has joined #openstack-qa11:37
*** wanghao has joined #openstack-qa11:37
openstackgerritAnn Kamyshnikova proposed a change to openstack/tempest: Verify more information in floating ip tests  https://review.openstack.org/6372311:38
*** SergeyLukjanov is now known as _SergeyLukjanov11:48
openstackrecheckopenstack/ceilometer change: https://review.openstack.org/63132 failed tempest because of: https://bugs.launchpad.net/bugs/125389611:48
*** _SergeyLukjanov has quit IRC11:48
openstackrecheckopenstack/heat change: https://review.openstack.org/63709 failed tempest because of: https://bugs.launchpad.net/bugs/125489011:57
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest because of: https://bugs.launchpad.net/bugs/125489012:02
*** prekarat has joined #openstack-qa12:21
openstackrecheckopenstack/cinder change: https://review.openstack.org/63720 failed tempest with an unrecognized error12:28
*** SergeyLukjanov has joined #openstack-qa12:30
*** che-arne has quit IRC12:37
*** jlibosva has quit IRC12:38
openstackgerritA change was merged to openstack/tempest: rename old config object to TempestConfigPrivate  https://review.openstack.org/6352912:40
openstackrecheckopenstack/tempest change: https://review.openstack.org/63530 failed tempest because of: https://bugs.launchpad.net/bugs/125389612:42
openstackrecheckopenstack/neutron change: https://review.openstack.org/53609 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/123543512:43
yfriedsdague: tempest gate doesn't have neutron-isolated like the neutron gate does, right?12:48
sdagueyfried: that's correct12:49
sdagueI think12:49
yfriedsdague: why is that? I want to push a change to isolated neutron test and I need to make sure I don't break the gate like I did during the weekend12:49
sdaguehistorical reasons12:50
yfriedsdague: or should I just push it and let neutron guys complain?12:50
sdaguebasically the neutron isolated jobs aren't fully isolated at this point12:50
sdaguethe whole end game is to get everything over to them12:51
*** ruhe is now known as ruhe_12:51
yfriedsdague: it did cause an issue last week...12:51
sdagueand drop the other jobs, but it moved *a lot* slower than we expected12:51
yfriedsdague: shouldn't we at least have this as a non-voting?12:51
sdagueyfried: I'd be fine with non-voting12:51
sdagueyou know how to add a job like that?12:51
*** ruhe_ has quit IRC12:52
yfriedsdague: no idea. I really have very little knowledge on the subject of the gate12:52
*** jlibosva has joined #openstack-qa12:52
sdaguesomething you want to learn? I can walk you through it. (need to grab a another cup of coffee first)12:53
*** yassine has joined #openstack-qa12:54
yfriedsdague: I would love to12:54
openstackrecheckopenstack/neutron change: https://review.openstack.org/59743 failed tempest because of: https://bugs.launchpad.net/bugs/125389612:54
sdagueso the key file that describes all of the zuul jobs is this - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml12:55
sdagueso you'll want to check yourself out an openstack-infra/config repo, which is the puppet repository that runs *all* of infrastructure12:56
yfriedsdague: would you like to have a bluejeans session?12:56
sdaguethe important part is it builds a matrix of jobs: x projects:12:56
sdaguebluejeans?12:57
yfriedsdague: http://bluejeans.com/12:57
yfriedsdague: or google hangouts12:57
yfriedsdague: or is this a quick explanation12:58
sdaguewell, I'm still very much early morning, so probably too early for video for me12:58
dkranzsdague: I can explain this to Yair12:58
sdagueI'd rather do it here, also this channel is logged, which is good for others learning12:58
yfriedsdague: ok. tnx12:58
yfriedI'm "listening"12:58
sdagueso the basic model is there is a list of jobs, and then you tell which project to run them on12:59
sdagueand which queue in that project to run them on12:59
sdagueso actually, I think for this case, we probably want to add the isolated jobs to the tempest experimental queue12:59
sdaguehttps://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml#L189212:59
sdaguewhich would let you run "check experimental" on any patch you think might be important for this13:00
sdagueand it will give you all those tests as an additional post of results13:00
*** ruhe has joined #openstack-qa13:01
sdagueand all you need to do to propose that is build a review with the job name in that experimental list13:01
openstackrecheckopenstack/python-cinderclient change: https://review.openstack.org/63147 failed tempest because of: https://bugs.launchpad.net/bugs/121048313:02
yfriedsdague: do you have an example of similar patch?13:07
yfrieddkranz: We will meet in ~50min, ok?13:09
dkranzyfried: ok13:10
sdagueyfried: https://review.openstack.org/#/c/53917/19/modules/openstack_project/files/zuul/layout.yaml is not exactly the same thing, but close13:15
sdagueit's just a single line patch with the check- jobs that yuo want to add to experimental13:15
yfriedsdague: I need to leave for a meeting shortly and it's still cloning (bad bandwidth). I will try this later. Thank you for the offer and the explanation13:16
openstackrecheckopenstack/nova change: https://review.openstack.org/59655 failed tempest with an unrecognized error13:16
openstackrecheckopenstack/neutron change: https://review.openstack.org/52930 failed tempest because of: https://bugs.launchpad.net/bugs/125489013:20
yfriedsdague: so I just need to add a line under "experimental"?13:23
yfriedsdague: will it run for every patch?13:24
openstackgerritSean Dague proposed a change to openstack-infra/devstack-gate: prune grenade service lists to things we can upgrade  https://review.openstack.org/6375113:26
dkranzyfried: No. Jobs in the experimental queue only run when you post a review comment of "check experimental" in a project that has stuff on its experimental queue.13:26
sdagueyfried: it won't run on every patch13:26
sdaguedkranz: if this looks good to you, could use another +2 - https://review.openstack.org/#/c/63649/13:27
dkranzsdague: Looking. Will get to more reviews around 10am13:28
sdagueyep, no worries13:28
sdaguemasauki and cyeoh put through the patch I really wanted to land, which was the TempestConfigPrivate one13:28
*** psharma has quit IRC13:29
sdaguesomeone else had manage to land an old style config change before that went in, so it needed a fix over the weekend, but all good now13:29
sdagueI also sent an email to the list13:29
dkranzsdague: ok, approved. Thanks for doing that.13:30
openstackrecheckopenstack/ceilometer change: https://review.openstack.org/59214 failed tempest because of: https://bugs.launchpad.net/bugs/125389613:31
dkranzsdague: I was going to reply suggesting that no one do this low-hanging-fruit in one commit13:31
dkranzsdague: Global bombings don't work so well in our environment13:32
*** prekarat has quit IRC13:32
sdaguedkranz: sure, be it could be a directory at a time13:32
*** jaypipes has joined #openstack-qa13:36
*** ChanServ sets mode: +o jaypipes13:36
openstackrecheckopenstack/nova change: https://review.openstack.org/63735 failed tempest because of: https://bugs.launchpad.net/bugs/125489013:37
openstackrecheckopenstack/keystone change: https://review.openstack.org/63722 failed tempest because of: https://bugs.launchpad.net/bugs/125389613:40
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/61102 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543513:41
openstackrecheckopenstack/neutron change: https://review.openstack.org/59743 failed tempest with an unrecognized error13:46
openstackrecheckopenstack/heat change: https://review.openstack.org/62836 failed tempest with an unrecognized error13:47
*** ruhe is now known as ruhe_13:51
*** ruhe_ has quit IRC13:52
*** yfried has quit IRC14:00
openstackgerritA change was merged to openstack/tempest: remove unneeded __init__ file  https://review.openstack.org/6353014:02
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Serialize plurals correctly in neutron xml client  https://review.openstack.org/5902914:05
*** ruhe has joined #openstack-qa14:06
openstackgerritA change was merged to openstack/tempest: remove last errant file parse warning  https://review.openstack.org/6364914:11
*** yfried has joined #openstack-qa14:13
openstackrecheckopenstack/python-novaclient change: https://review.openstack.org/62550 failed tempest with an unrecognized error14:19
*** wanghao has quit IRC14:23
*** wanghao has joined #openstack-qa14:23
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/63751 failed tempest with an unrecognized error14:25
*** macjack has quit IRC14:28
openstackrecheckopenstack/neutron change: https://review.openstack.org/52930 failed tempest because of: https://bugs.launchpad.net/bugs/125489014:37
openstackrecheckopenstack/keystone change: https://review.openstack.org/63722 failed tempest because of: https://bugs.launchpad.net/bugs/1250168 and https://bugs.launchpad.net/bugs/125489014:38
*** rossella_s has joined #openstack-qa14:41
*** malini has joined #openstack-qa14:50
openstackrecheckopenstack/neutron change: https://review.openstack.org/62696 failed tempest because of: https://bugs.launchpad.net/bugs/125389614:50
openstackgerritA change was merged to openstack-infra/devstack-gate: Always dump errors to console  https://review.openstack.org/6185014:53
*** rwsu has joined #openstack-qa14:59
*** rustlebee is now known as russellb14:59
openstackrecheckopenstack/glance change: https://review.openstack.org/59725 failed tempest because of: https://bugs.launchpad.net/bugs/125389615:00
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Serialize plurals correctly in neutron xml client  https://review.openstack.org/5902915:02
*** ruhe has quit IRC15:07
openstackrecheckopenstack/neutron change: https://review.openstack.org/59743 failed tempest because of: https://bugs.launchpad.net/bugs/125489015:08
openstackrecheckopenstack/glance change: https://review.openstack.org/63655 failed tempest because of: https://bugs.launchpad.net/bugs/125389615:09
openstackrecheckopenstack/glance change: https://review.openstack.org/62231 failed tempest because of: https://bugs.launchpad.net/bugs/1259907 and https://bugs.launchpad.net/bugs/125389615:11
malinihello!!! I am working on adding marconi tests to tempest. I have a jenkins failure coz it cannot find the queuing endpoint (http://logs.openstack.org/49/63449/3/check/check-tempest-dsvm-full/a525b64/ ) .  My guess is its because marconi is not enabled in devstack that gating uses. Any pointers to what I need to update to make marconi service enabled ?15:15
openstackrecheckopenstack/glance change: https://review.openstack.org/63727 failed tempest with an unrecognized error15:21
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Use create_router_interface and create_port  https://review.openstack.org/6152715:26
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Serialize plurals correctly in neutron xml client  https://review.openstack.org/5902915:26
openstackrecheckopenstack/nova change: https://review.openstack.org/63566 failed tempest with an unrecognized error15:28
*** jlibosva has quit IRC15:30
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Add create_floating_ip function  https://review.openstack.org/6152915:33
openstackrecheckopenstack/nova change: https://review.openstack.org/63769 failed tempest because of: https://bugs.launchpad.net/bugs/125489015:34
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Add 2 tests to the Floating IP test case  https://review.openstack.org/5634915:35
openstackrecheckopenstack/neutron change: https://review.openstack.org/59359 failed tempest because of: https://bugs.launchpad.net/bugs/125489015:37
openstackrecheckopenstack/requirements change: https://review.openstack.org/63754 failed tempest because of: https://bugs.launchpad.net/bugs/125990715:38
*** SergeyLukjanov is now known as _SergeyLukjanov15:42
*** _SergeyLukjanov has quit IRC15:43
openstackgerritRossella Sblendido proposed a change to openstack/tempest: Add negative tests for routers  https://review.openstack.org/5668015:48
vponomaryovmalini: marconi is not installed in job you mentioned and you have enabled your tests by default. As you can see, there is no ironic and savanna too in this jobs15:48
malinivponomaryov: is the standard to disable the tests till the project graduates?15:49
malinivponomaryov: I am trying to figure out what the usual process is to add a new project to tempest15:50
*** rossella_s_ has joined #openstack-qa15:51
vponomaryovI dont have such info, I recommend you to look how is it done in other projects15:51
malinithanks for the pointer vponomaryov!!15:51
openstackrecheckopenstack/requirements change: https://review.openstack.org/63770 failed tempest because of: https://bugs.launchpad.net/bugs/125389615:51
*** rossella_s has quit IRC15:52
*** rossella_s_ is now known as rossella_s15:52
*** pmathews has joined #openstack-qa15:53
openstackgerritMalini Kamalambal proposed a change to openstack/tempest: Adds basic Marconi test  https://review.openstack.org/6344915:56
*** yfried has quit IRC15:56
*** yfried has joined #openstack-qa15:57
*** corvus is now known as jeblair15:58
vponomaryovmalini: if you want change value of some opt, you should change it in tempest/config.py and after it regenarate, if I am not wrong. Because if someone else change it, your value become as in  tempest/config.py16:01
vponomaryovit is related to your patch set 416:02
malinithx vponomaryov..I updated https://review.openstack.org/#/c/63449/4/etc/tempest.conf.sample manually to set service_available.queuing to false with this patch..Apparently tht didnt help16:03
*** SergeyLukjanov has joined #openstack-qa16:03
maliniI will chk the config.py16:04
openstackrecheckopenstack/neutron change: https://review.openstack.org/63775 failed tempest because of: https://bugs.launchpad.net/bugs/121048316:07
*** yfried has quit IRC16:08
openstackgerritMalini Kamalambal proposed a change to openstack/tempest: Adds basic Marconi test  https://review.openstack.org/6344916:08
openstackrecheckopenstack/tempest change: https://review.openstack.org/61527 failed tempest because of: https://bugs.launchpad.net/bugs/125389616:08
openstackrecheckopenstack/nova change: https://review.openstack.org/63084 failed tempest with an unrecognized error16:13
openstackgerritMalini Kamalambal proposed a change to openstack/tempest: Adds basic Marconi test  https://review.openstack.org/6344916:15
openstackrecheckopenstack/neutron change: https://review.openstack.org/59743 failed tempest because of: https://bugs.launchpad.net/bugs/125489016:22
openstackrecheckopenstack/neutron change: https://review.openstack.org/58954 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543516:23
openstackrecheckopenstack/neutron change: https://review.openstack.org/61067 failed tempest because of: https://bugs.launchpad.net/bugs/125489016:27
*** SergeyLukjanov has quit IRC16:28
*** ruhe has joined #openstack-qa16:33
*** ruhe has quit IRC16:34
*** prekarat has joined #openstack-qa16:44
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/63751 failed tempest because of: https://bugs.launchpad.net/bugs/125389616:44
*** rossella_s_ has joined #openstack-qa16:48
*** rossella_s has quit IRC16:49
*** rossella_s_ is now known as rossella_s16:49
openstackrecheckopenstack/neutron change: https://review.openstack.org/63775 failed tempest with an unrecognized error16:52
*** pmathews has quit IRC16:55
*** rossella_s has quit IRC16:58
*** prekarat has quit IRC16:59
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/63619 failed tempest because of: https://bugs.launchpad.net/bugs/125489017:01
*** cyeoh has quit IRC17:03
*** rossella_s has joined #openstack-qa17:05
*** mlavalle has joined #openstack-qa17:11
*** yassine has quit IRC17:16
*** pmathews has joined #openstack-qa17:17
*** pmathews has quit IRC17:22
openstackgerritBob Ball proposed a change to openstack-dev/devstack: XenAPI: Fix bug with Xen ext4-using guests  https://review.openstack.org/6379517:25
*** akamyshnikova_ has quit IRC17:27
*** metabro has quit IRC17:27
*** vrovachev has quit IRC17:27
*** obondarev has quit IRC17:27
*** enikanorov has quit IRC17:27
*** akamyshnikova_ has joined #openstack-qa17:28
*** metabro has joined #openstack-qa17:28
*** enikanorov has joined #openstack-qa17:28
*** vrovachev has joined #openstack-qa17:29
*** obondarev has joined #openstack-qa17:30
*** yfried has joined #openstack-qa17:37
*** safchain has quit IRC17:42
*** ruhe has joined #openstack-qa17:47
*** harlowja has joined #openstack-qa17:50
openstackrecheckopenstack/neutron change: https://review.openstack.org/60571 failed tempest with an unrecognized error17:51
*** dkorn has quit IRC17:51
openstackrecheckopenstack/cinder change: https://review.openstack.org/59334 failed tempest because of: https://bugs.launchpad.net/bugs/125389617:51
*** SergeyLukjanov has joined #openstack-qa17:52
*** rossella_s_ has joined #openstack-qa17:52
openstackgerritA change was merged to openstack-infra/devstack-gate: Timeout devstack/tempest internally  https://review.openstack.org/6351217:53
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/62983 failed tempest because of: https://bugs.launchpad.net/bugs/125389617:53
*** rossella_s has quit IRC17:54
*** rossella_s has joined #openstack-qa17:55
*** rossella_s_ has quit IRC17:57
openstackrecheckopenstack/requirements change: https://review.openstack.org/63754 failed tempest because of: https://bugs.launchpad.net/bugs/125389617:57
openstackrecheckopenstack/neutron change: https://review.openstack.org/61168 failed tempest because of: https://bugs.launchpad.net/bugs/125489017:58
*** tkammer has quit IRC18:00
openstackrecheckopenstack/cinder change: https://review.openstack.org/61142 failed tempest with an unrecognized error18:00
openstackrecheckopenstack/neutron change: https://review.openstack.org/58533 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/1235435 and https://bugs.launchpad.net/bugs/125389618:14
*** rossella_s_ has joined #openstack-qa18:17
openstackrecheckopenstack/neutron change: https://review.openstack.org/59948 failed tempest because of: https://bugs.launchpad.net/bugs/125489018:18
openstackrecheckopenstack/heat change: https://review.openstack.org/58641 failed tempest because of: https://bugs.launchpad.net/bugs/125389618:19
*** rossella_s has quit IRC18:19
*** rossella_s_ is now known as rossella_s18:19
*** ruhe has quit IRC18:28
*** markmcclain has joined #openstack-qa18:46
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/63619 failed tempest because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1251448 and https://bugs.launchpad.net/bugs/123543518:48
*** rossella_s_ has joined #openstack-qa18:51
*** DennyZhang has joined #openstack-qa18:51
*** rossella_s has quit IRC18:52
*** rossella_s_ is now known as rossella_s18:52
openstackgerritA change was merged to openstack-infra/devstack-gate: Remove strange duplication of setup_host and output  https://review.openstack.org/6298318:53
mlavallemtreinish: ping18:56
*** rossella_s_ has joined #openstack-qa18:59
*** rossella_s has quit IRC19:02
*** rossella_s_ is now known as rossella_s19:02
sdaguemlavalle: he's off on vacation for the rest of the year19:03
mlavallesdague: it seems I missed the conversation on negative api tests19:04
mlavallewe are not implementing those anymore?19:04
sdagueas a policy, not new ones, because we really need to do this in a generative way19:06
sdaguehand writing out the negative space is no good, it's too much time, and too much review time19:06
sdagueanyway, it's basically holidays, so probably a conversation we won't really pick up until Jan19:06
mlavalleso' I've doing the gap analysis for neutron api tests. I've been including negative api tests19:07
mlavalleshould I continue doing that?19:07
openstackrecheckopenstack/neutron change: https://review.openstack.org/59948 failed tempest with an unrecognized error19:08
*** rossella_s_ has joined #openstack-qa19:12
*** rossella_s has quit IRC19:14
*** rossella_s_ is now known as rossella_s19:14
*** DennyZhang has quit IRC19:17
*** DennyZhang has joined #openstack-qa19:17
*** yamahata__ has joined #openstack-qa19:19
*** hemna_ has joined #openstack-qa19:19
sdaguemlavalle: so we should fix where it's disjoint to nova-network19:20
*** vrovachev1 has joined #openstack-qa19:20
sdaguebut I wouldn't focus on adding a lot beyond that19:20
mlavallecool19:20
mlavallesdague: to make sure we are in alignment, let me give you an example19:21
mlavallelbaas….. that doesn't exist in nova network, right? so we should develop negative tests for that. Am I correct?19:22
sdaguemlavalle: no, we shouldn't at this point19:23
mlavallesdague: ok19:23
sdaguenegative tests for new things need to come in via the generative approach19:23
mlavalleI'll concentrate on positive tests19:23
sdagueyep, that would be good19:23
mlavallesdague: are we meeting on Thursday/19:24
mlavalle?19:24
sdagueno, skipping the next 2 meetings. Did mtreinish not send that to the list?19:24
mlavalleprobably he did and I just missed it….. happy Holidays!19:25
sdagueyou too19:25
*** vrovachev has quit IRC19:26
*** openstackrecheck has quit IRC19:26
*** yamahata_ has quit IRC19:26
*** hemnafk has quit IRC19:26
*** openstackrecheck has joined #openstack-qa19:26
*** rossella_s_ has joined #openstack-qa19:34
*** rossella_s has quit IRC19:38
*** rossella_s_ is now known as rossella_s19:38
*** DennyZhang has quit IRC19:44
openstackrecheckopenstack/cinder change: https://review.openstack.org/63560 failed tempest because of: https://bugs.launchpad.net/bugs/125389619:55
openstackgerritmouad benchchaoui proposed a change to openstack/tempest: Make testing neutron extensions optional  https://review.openstack.org/5513320:03
*** malini is now known as malini_afk20:09
jog0sdague: I plan on adding rolling nova-compute support to grenade this week20:15
openstackgerritMauro S. M. Rodrigues proposed a change to openstack/tempest: Add aggregates scenario test  https://review.openstack.org/6261820:22
jog0and when you have a minute wanted to sync on the steps needed20:23
sdaguesure20:25
sdagueyou want to sync here, or do like a g+ thing?20:29
*** rossella_s_ has joined #openstack-qa20:32
openstackrecheckopenstack/neutron change: https://review.openstack.org/63775 failed tempest with an unrecognized error20:33
jog0here should be fine, at least as a first pass20:33
*** rossella_s has quit IRC20:34
*** rossella_s_ is now known as rossella_s20:34
sdagueso basically the trick of it is going to be convincing grenade to start services off different trees after upgrade20:35
sdagueso you can use the whole "old" piece20:35
jog0sdague: right20:35
jog0so I think we want venvs for new and old python deps20:35
sdaguebecause the assumption is to simulate that this was different nodes?20:38
jog0sdague: and if my memory serves me correctly we want to ran one grenade as is, and another that upgrades everything but nova-compute20:38
jog0sdague: because the assumption is that $stable and $trunk may have different dependencies20:39
sdaguesure20:39
sdaguebut if you don't actually shut down stable, does it matter?20:40
sdagueso I'd actually start with this - https://github.com/openstack-dev/grenade/blob/master/stop-base#L3620:40
sdagueand make stop more granular20:40
jog0I don't *think* so but that sounds very hackey20:40
sdaguebecause right now it's just bouncing the whole screen session20:40
sdaguejog0: why, it's repeatable20:40
jog0are we 100% sure upgrading the deps from underneath python won't hurt something that is running?20:41
sdaguethis is going to take a while to figure out the right way to do this, so expect us to bounce around20:41
jog0what happens if there is a lazy load20:41
jog0sdague: but yeah I agree we can start without the venv and add it in20:41
sdaguewell, I think we then isolate it with venv after wards20:42
sdagueso start with the stop list being something you can control20:42
jog0sounds good20:42
sdagueand the logic to start up only the listed services on the new side20:42
sdaguethat will get you about 90% of the way there20:42
sdaguejog0: how much are you working between now an new years?20:43
*** rossella_s_ has joined #openstack-qa20:43
jog0sdague: almost full -- taking vacation in January instead20:43
sdaguewell... good luck on reviews :)20:44
jog0I don't expect to get any code reviewed ... just be able to get some work done20:44
openstackrecheckopenstack/tempest change: https://review.openstack.org/55133 failed tempest because of: https://bugs.launchpad.net/bugs/125489020:45
jog0I think first step should be to wire up infra with an experimental job20:45
jog0that has the stop list20:45
*** rossella_s has quit IRC20:47
*** rossella_s_ is now known as rossella_s20:47
*** pmathews has joined #openstack-qa20:47
jog0https://etherpad.openstack.org/p/grenade-rolling-upgrades20:48
sdaguecool, seems sane20:49
jog0and this is explicitly constrainted in scope, as the big upgrade test will be coming from tripleo20:51
sdagueyep20:54
jog0sdague: what are your thoughts un expanding the set of tempest tests that grenade runs?20:54
sdague-120:54
jog0you know a the worst kind of feedback is a -1 without an explination heh20:55
sdagueit's coming...20:55
*** dkorn has joined #openstack-qa20:55
sdagueso if we weren't going to get lynched for it, I'd actually want to run the full battery20:55
sdaguebut the time is a bit too long for that20:56
sdaguethe whole point is that it should be a working openstack cloud20:56
sdagueso it should pass all the tests20:56
jog0what about full api but no scenario or something20:56
openstackrecheckopenstack/neutron change: https://review.openstack.org/61067 failed tempest with an unrecognized error20:56
jog0or vice versa20:56
sdaguewhy do you want to decrease the tests?20:57
jog0I don't -- I want to get the most number of meaningful tests in the least amount of time that covers the things we expect to fail20:58
jog0not sure what that is though20:58
sdaguehonestly, we never know what we expect to fail20:58
sdagueand everything we don't tests, seems to eventually fail20:58
sdaguesee: neutron + devstack exercises20:59
jog0right ...20:59
jog0I don't see any good answer besides test it all20:59
sdagueyeh, the current smoke compromise is what it is20:59
jog0why is greande so slow anyway?20:59
sdaguesmoke is currently all scenario tests + tagged api tests21:00
sdague*so slow*?21:00
*** amotoki_ has joined #openstack-qa21:00
sdagueit needs to sanity check the first cloud with a set of tests to make sure that it stood up, otherwise, debugging the fails later are terrible21:02
sdaguefeel free to comb logs and look for optimizations21:02
jog0https://jenkins02.openstack.org/view/All/job/gate-grenade-dsvm/buildTimeTrend21:04
jog0its about 10 min faster then https://jenkins02.openstack.org/job/gate-tempest-dsvm-postgres-full/buildTimeTrend21:04
sdagueyep21:04
sdagueremember, real testing takes real time :)21:05
jog0interesting so grenade setup is about 10 min slower then devstack21:05
jog0not too bad21:05
sdagueso I take it you aren't coming to montreal any more?21:05
jog0431 seconds of tempest in grenade vs 1635 in full21:06
openstackrecheckopenstack/python-neutronclient change: https://review.openstack.org/63619 failed tempest because of: https://bugs.launchpad.net/bugs/125489021:06
jog0sdague: no, too much traveling21:06
sdagueyeh, I can understand that21:06
jog0I will see anteaya the week before though21:06
*** amotoki is now known as __amotoki__21:06
*** amotoki_ is now known as amotoki21:06
sdaguedkranz: if you have a few minutes, curious on thoughts on this style approach - https://review.openstack.org/#/c/63540/21:08
dkranzsdague: Looking...21:08
openstackgerritA change was merged to openstack-infra/devstack-gate: prune grenade service lists to things we can upgrade  https://review.openstack.org/6375121:08
openstackgerritA change was merged to openstack/tempest: Add swift scenario tests  https://review.openstack.org/6184521:08
sdagueit's in no way complete, but just a sample of some new decorators to try to get all the skip logic out of the tests themselves21:09
dkranzsdague: It seems reasonable to me, though I might have used "protocol" rather than "payload".21:11
*** pmathews has quit IRC21:11
sdaguedkranz: sure, exact names I can easily adjust21:12
dkranzsdague: Right21:12
sdaguejust wanted to sanity check it before I officially stop working for the year, in case I decide I want to play with it when out21:12
sdagueso it's not a waste of time21:12
dkranzsdague: Sure, though I am only one opinion :)21:13
dkranzsdague: Can we discuss https://review.openstack.org/#/c/61978 a bit?21:13
dkranzsdague: My first reaction, and second, is ick21:13
sdaguedkranz: well, how do we expect clients to do retry?21:13
dkranzsdague: I get the motivation but the bug is reallyin the clients21:13
dkranzsdague: swiftclient actually does a kind of retry21:14
dkranzsdague: I guess if we have to do this I would prefer to have a separate call for each client and a function that takes arguments like the code to check for, name of methoc, etc.21:15
dkranzsdague: Rather than one giant mess that will break when anything changes or new clients are added.21:15
dkranzsdague: Oh well.21:15
dkranzsdague: And do we want this on by default, and in the gate?21:16
sdaguewe turn off rate limit in the gate21:16
sdagueso we don't need it there21:16
sdaguedkranz: well is it something where there would be reasonable unit tests for it?21:17
sdagueso we could test that it actually works21:17
sdaguein the unit test path21:17
dkranzsdague: There could be21:17
sdagueI agree it's black magic enough that I'm twitchy on it21:17
sdaguewhich is why I didn't cycle around and +2 it yet21:17
notmynameseems that I may want to pay attention to this conversation, butI can't seem to find the beginning to see what it's about21:18
dkranzsdague: Well we can wait until the new year21:18
dkranznotmyname: It is about working around the fact that the python clients don't let you work around rate limit errors without catching them at every call.21:18
dkranznotmyname: Though swiftclient had some retry logic21:18
dkranznotmyname: But not specific for this case.21:19
notmynamedkranz: https://github.com/openstack/python-swiftclient/blob/master/swiftclient/client.py#L111421:19
notmynamedkranz: not specific to this case? how do you mean?21:19
dkranznotmyname: Yes, that code. I meant that it retries on all errors, not just ratelimit.21:20
notmynameat least with python-swiftclient, errors will result in retries (including rate limit errors)21:20
notmynameah, ok21:20
dkranznotmyname: I think that could be improved21:20
openstackrecheckopenstack/tempest change: https://review.openstack.org/62618 failed tempest with an unrecognized error21:20
dkranznotmyname: But the other clients don't do anything at all21:20
notmynamesomething we should change in the swift client? what are you looking for?21:20
notmynamedkranz: that seems like a bug with those clients21:20
*** vrovachev has joined #openstack-qa21:21
dkranznotmyname: Indeed https://bugs.launchpad.net/python-keystoneclient/+bug/126306921:21
dkranznotmyname: I think this needs more thought and/or discussion21:21
dkranznotmyname: Sometimes you want a call to just wait until it can get in. Sometimes you need to react to the rate limiting.21:22
dkranznotmyname: My thought was just that it would be nice if when you make the connection you could specify that.21:22
dkranznotmyname: It is what _retry does but just more general21:23
dkranznotmyname: But i'm not sure how much generality is needed.21:23
vrovachevGuys, please consider this patch https://review.openstack.org/#/c/61958/.  It's very important to telemetry tempest tests.21:23
notmynameactually, _retry is part of the Connection class, so the numner of retries is settable per connection. and if you don't want anything, then you can using the direct module methods (which Connection() instances just wrap in _retry())21:24
dkranznotmyname: Yes, you could do that I suppose.21:25
dkranznotmyname: I'll think if that solves the tempest problem.21:25
dkranznotmyname: But it would only help for swift, which just merged scenario tests.21:25
dkranzvrovachev: You need to take this to the #openstack-infra channel. Most of us do not have +2 for devstack-gate.21:26
notmynamedkranz: the tempest problem is that client libs either (1) don't do any retries or (2) do retries for everything, both errors and rate limits?21:26
notmynamedkranz: and swift is in category 2?21:26
dkranznotmyname: All but switfclient don't do any, swift is (2)21:26
dkranznotmyname: Yes21:26
dkranznotmyname: So this awful monkeypatch stuff was proposed and I just was trying to come up with a different way.21:27
sdaguenotmyname: right, which is not a problem in the gate. But for people that want to hit live clouds with rate limits... it can be.21:27
vrovachevdkranz: thx for info :)21:27
shardydkranz: Thanks for the reviews, I put a response on https://review.openstack.org/#/c/61175/21:28
openstackrecheckopenstack/nova change: https://review.openstack.org/57747 failed tempest because of: https://bugs.launchpad.net/bugs/125489021:29
shardydkranz: I'd like to keep some method of testing delete in there, but I'm happy to post a refactor patch which abstracts it into a tearDown handler, or whatever you prefer21:29
dkranzshardy: I would actually be ok with just a comment that it is intended to happen before cleanup.21:30
dkranzshardy: Just not obvious :)21:30
shardydkranz: Ok, sure, should I add it to that patch, or would you like a comment added to every test?21:30
shardydkranz: maybe the cleanest way would be to move the delete to a common teardown handler, and put a comment there21:31
dkranzshardy: To me it is only needed where there is a delete call that is not needed to actually make sure things are cleaned up21:31
openstackrecheckopenstack/nova change: https://review.openstack.org/57748 failed tempest with an unrecognized error21:31
dkranzshardy: That would be ok too.21:31
openstackrecheckopenstack/neutron change: https://review.openstack.org/63056 failed tempest with an unrecognized error21:32
shardydkranz: Ok, I'll post a followup patch which does that, thanks!21:32
*** vrovachev has left #openstack-qa21:37
openstackgerritSean Dague proposed a change to openstack/tempest: allow hypervisors to be down but still pass  https://review.openstack.org/6187921:39
notmynamedkranz: ok, so clarification, python-swiftclient doesn't automatically retry on ratelimit errors21:41
dkranznotmyname: That is my reading of the code21:42
openstackgerritSean Dague proposed a change to openstack/tempest: wip: use getattr to cleanup config further  https://review.openstack.org/6383721:47
*** jamielennox|away is now known as jamielennox21:47
*** mestery has joined #openstack-qa21:50
rossella_sdkranz: hi! thanks for reviewing my patches. I have on questions, who can approve them?21:51
notmynamedkranz: https://review.openstack.org/#/c/63838/21:51
dkranznotmyname: That is along the lines I was thinking21:53
openstackgerritVadim Rovachev proposed a change to openstack-infra/devstack-gate: Added ceilometer-anotification to enabled services  https://review.openstack.org/6195821:54
notmynamedkranz: if you could, I'd appreciate you leaving a comment about why you want the change21:54
dkranznotmyname: Sure thing.21:54
notmynamethanks21:54
openstackrecheckopenstack/nova change: https://review.openstack.org/63827 failed tempest with an unrecognized error21:55
dkranzrossella_s: Any one on the core tempest team. A bunch of folks are kindof out for the holidays. Hopefully one of our Japanese core reviewers can get to them.21:56
rossella_sdkranz: thanks! I am gonna be out soon too. I wanted to get them merged before my holidays.21:57
*** dkorn has quit IRC21:57
rossella_sdkranz: can you please tell me who I can ping to get an approved?21:58
rossella_ss/approved/approval21:58
dkranzrossella_s: Maybe sdague can get to it before he leaves.21:58
rossella_sthanks a lot dkranz!21:59
openstackrecheckopenstack/heat change: https://review.openstack.org/62441 failed tempest with an unrecognized error21:59
dkranzrossella_s: Ken'ichi Ohmichi is in Japan but I'm not sure of his irc handle. He is not here now.22:00
rossella_sdkranz: thanks again, I'll ping sdague22:01
rossella_ssdague: will you be so kind to have a look at my patches? It's almost xmas :) https://review.openstack.org/#/c/59029/  https://review.openstack.org/#/c/56349/ https://review.openstack.org/#/c/56680/ https://review.openstack.org/#/c/61527/ https://review.openstack.org/#/c/61529/  thanks!!!22:01
openstackgerritSteven Hardy proposed a change to openstack/tempest: keystone trusts API test, move delete into tearDown handler  https://review.openstack.org/6384122:02
openstackgerritSteven Hardy proposed a change to openstack/tempest: keystone OS-TRUST extension, test list operations  https://review.openstack.org/6117622:02
openstackgerritSteven Hardy proposed a change to openstack/tempest: keystone OS-TRUST extension, test with expiry  https://review.openstack.org/6117522:02
shardydkranz: ^^22:02
dkranzshardy: thanks22:03
openstackrecheckopenstack/heat change: https://review.openstack.org/62988 failed tempest with an unrecognized error22:03
rossella_sdkranz: sorry to bother again. Can you have a look at https://review.openstack.org/#/c/61527/ https://review.openstack.org/#/c/61529/ ? I've realized you didn't check those too. Thanks a lot!!22:04
dkranznotmyname: ok, I hope my comment is sufficient.22:04
*** shardy is now known as shardy_afk22:04
notmynamedkranz: perfect, thanks22:05
bnemecOh ugh.  This commit https://github.com/openstack/tempest/commit/87f772c0bd59790aa43de59938d9c138963f9f4e means that a basic "testr run foo" doesn't work anymore.  At least not for me.22:05
bnemecThat was really unpleasant to figure out.22:05
bnemecgit bisect FTW though. :-)22:05
*** amotoki has quit IRC22:06
dkranzsdague:    ^^^22:06
*** rossella_s has quit IRC22:08
openstackrecheckopenstack/heat change: https://review.openstack.org/62760 failed tempest because of: https://bugs.launchpad.net/bugs/122451822:09
bnemecNow why does it work when I run 'OS_TEST_PATH=./tempest/test_discover testr run tempest.api.compute.*' but not if I change the default in .testr.conf...22:10
sdaguedkranz: so there is this alternative approach - https://review.openstack.org/#/c/62614/22:12
sdagueI'm concerned they will collide22:12
dkranzsdague: What will collide with that?22:13
openstackrecheckopenstack/keystone change: https://review.openstack.org/63570 failed tempest because of: https://bugs.launchpad.net/bugs/125389622:13
sdaguedkranz: oh, you are right22:14
sdaguenever mind, the earlier patch went through22:14
sdaguehonestly, my brain is a bit too fried to do code reviews at this point22:14
dkranzsdague: ok, understood22:14
dkranzsdague: I have spent more time doing that today than any other day in a while. I'm about ready to quit too.22:15
dkranzsdague: Maybe just one last one, since you made a comment that was not understood, and I'm not quite sure either https://review.openstack.org/#/c/61529/22:19
*** che-arne has joined #openstack-qa22:24
openstackrecheckopenstack/tempest change: https://review.openstack.org/61879 failed tempest with an unrecognized error22:26
*** mlavalle has quit IRC22:37
*** pmathews has joined #openstack-qa22:37
*** HenryG has quit IRC22:38
*** HenryG has joined #openstack-qa22:38
*** HenryG has quit IRC22:43
*** HenryG has joined #openstack-qa22:44
sdaguedkranz: so it's a little weird to leave those ips around for class teardown instead of just cleaning them up with an addCleanup22:44
openstackgerritBen Nemec proposed a change to openstack/tempest: Fix OS_TEST_PATH in .testr.conf  https://review.openstack.org/6384722:49
bnemec^Victory!22:49
sdaguebnemec: did you really put html in the commit message....22:56
sdagueಠ_ಠ22:56
bnemecsdague: No, I definitely did not.  I'm trying to figure out what's going on there.22:56
bnemecIn my local git log it just show http://stackoverflow.com/questions/10938483/bash-specifying-environment-variables-for-echo-on-command-line22:57
clarkblol22:58
clarkbits gerrit22:58
clarkbwe have a fixed in place for that problem, but it requires restarting gerrit22:58
clarkbfungi: ^ maybe we should do that today?22:58
fungiclarkb: sure thing. looking at the graph to see if it's a good time22:59
openstackgerritmouad benchchaoui proposed a change to openstack/tempest: Make testing neutron agents optional  https://review.openstack.org/5513323:00
bnemecPhew, so it wasn't me. :-)23:00
clarkbfungi: and probably double check jhesketh's change is in place and all that23:00
fungilooks like activity is starting to trail off for review.o.o23:02
clarkbfungi: I do need to step out shortly. Doing last minute holdiay stuff (woo!) and probably grabbing food23:02
sdaguefungi: yeh, it's been a slow day23:03
fungipuppet is running on review, gerrit.config has <p></p> in the commentlink pattern23:04
clarkbfungi: and uses html instead of link?23:04
sdaguebnemec: so why does it need to come after the python command?23:04
fungiclarkb: yep23:05
sdague23:06
bnemecsdague: If you try to set it on the command-line the way it was, it doesn't take because the variables are evaluated before the environment is altered by the variables set on the command-line.23:07
bnemecThe stackoverflow link is basically a simple example of that.23:07
bnemecsdague: If you run "FOO=foo echo $FOO" in a shell you'll see that it doesn't work the way you expect.  Hopefully. :-)23:10
sdagueyep, gotcha23:10
sdaguetook a second to think it through23:10
sdagueyeh, this is actually the fix for the thing beagles and I were hitting earlier23:11
sdaguenice23:11
*** pmathews has quit IRC23:12
sdaguedkranz: so I looked a rossa's patches, +Aed 2 of them in23:16
sdaguedkranz: https://review.openstack.org/#/c/63847/ should get approved if you are still kicking around23:16
openstackrecheckopenstack/neutron change: https://review.openstack.org/63840 failed tempest with an unrecognized error23:20
sdagueok, officially done with work for the year, though I might check on some things in the early morning hours over the holiday. Happy holidays everyone.23:20
*** markmcclain has quit IRC23:24
*** markmcclain has joined #openstack-qa23:26
*** harlowja_ has joined #openstack-qa23:29
*** harlowja has quit IRC23:29
openstackgerritA change was merged to openstack-dev/devstack: Neutron: create network resources when agents are started  https://review.openstack.org/6364123:34
openstackrecheckopenstack/nova change: https://review.openstack.org/63827 failed tempest with an unrecognized error23:37
openstackgerritA change was merged to openstack-infra/devstack-gate: Added ceilometer-anotification to enabled services  https://review.openstack.org/6195823:43
*** SergeyLukjanov has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!