| openstackrecheck | Console logs not available after 14:02s for check-tempest-dsvm-full 98828,3,f5bbad3 | 00:03 |
|---|---|---|
| adam_g | hey all. wondering why we can't seem to get E-R to comment on matched queries in the ironic gate? bug #1326754 shows up #3 @ http://status.openstack.org/elastic-recheck/ but the bot is silent on reviews that fail on it | 00:06 |
| *** mmaglana has quit IRC | 00:13 | |
| openstackrecheck | Console logs not available after 13:26s for gate-python-glanceclient-python27 95233,4,9c9b5ff | 00:20 |
| *** weshay has joined #openstack-qa | 00:22 | |
| *** matsuhashi has joined #openstack-qa | 00:27 | |
| mgagne | adam_g: could be logstash is falling behind in indexing all the logs, that's the reason I heard on infra a couple of days ago: "<clarkb> sdague: we are very behind on indexing last I checked" | 00:31 |
| openstackgerrit | A change was merged to openstack/tempest: Make run_stress.py script an entry point https://review.openstack.org/92519 | 00:31 |
| *** jaypipes has quit IRC | 00:32 | |
| openstackrecheck | Console logs not available after 13:23s for gate-keystone-pep8 98836,1,5d86f46 | 00:33 |
| cyeoh | mtreinish: hi - are you around? | 00:35 |
| *** dmorita has joined #openstack-qa | 00:36 | |
| mgagne | adam_g: or https://bugs.launchpad.net/openstack-ci/+bug/1328318 =) | 00:37 |
| adam_g | mgagne, ah yeah. :) | 00:37 |
| mgagne | adam_g: since you are there, who should I bug regarding ubuntu packaging? | 00:39 |
| adam_g | mgagne, #ubuntu-server. i may be able to answer questions, but i haven't been active there in a while | 00:39 |
| mgagne | adam_g: alright, I'll bug them then | 00:40 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/96829 | 00:40 |
| *** mmaglana has joined #openstack-qa | 00:43 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron 97529,7,18a9d63 | 00:46 |
| *** yamahata has quit IRC | 00:52 | |
| openstackrecheck | Console logs not available after 13:36s for gate-tempest-dsvm-neutron 98600,2,30490e7 | 01:00 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/elastic-recheck: Broaden 1326312 to match py26/py27 failures https://review.openstack.org/98929 | 01:01 |
| mtreinish | cyeoh: yeah what's up? | 01:04 |
| cyeoh | mtreinish: oh just wanted to chat about https://review.openstack.org/#/c/97834/3 | 01:09 |
| mtreinish | sure | 01:10 |
| cyeoh | so I'm happy with it from the POV re: thats what is being enforced at the moment | 01:10 |
| cyeoh | but I was wondering what you thought about having an experimental tag for tests which test the API which we deliberately believe are not yet stable across releases | 01:10 |
| cyeoh | and that the experimental tag would mean that its only run against the latest project release? | 01:11 |
| mtreinish | so that's where I think we'll be going for the experimental api tests we have in tree now | 01:11 |
| mtreinish | but for future apis I think that marun's neutron thing is really the way it should be done | 01:12 |
| mtreinish | and then it'll be a write once graduation into a tempest test when the api is stabalized | 01:12 |
| cyeoh | so I think I missed marun's neutron thing? | 01:12 |
| mtreinish | oh, I left that as a comment on https://review.openstack.org/#/c/98230/ | 01:13 |
| cyeoh | is he doing tempest like testing within the neutron project? | 01:13 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/elastic-recheck: Broaden 1286635 https://review.openstack.org/98931 | 01:13 |
| marun | mtreinish: I've been quiet post-summit but I'll be talking to markmcclain this week about ensuring the neutron core refactor provides for a mechanical schema. | 01:13 |
| mtreinish | yeah pretty much | 01:13 |
| cyeoh | hrm, so I suspect that's a bad idea in general | 01:13 |
| marun | hah | 01:14 |
| cyeoh | marun: no offence to you at all, in fact its a complement :-) | 01:14 |
| openstackrecheck | Console logs not available after 13:45s for check-tempest-dsvm-neutron 89474,13,efe8db2 | 01:14 |
| mtreinish | cyeoh: it reuses the tempest clients the etherpad is here: https://etherpad.openstack.org/p/juno-qa-functional-api | 01:14 |
| marun | cyeoh: well, the goal is to _develop_ tests in tree | 01:14 |
| cyeoh | so we've seen in the past for example API backwards incompatible changes made and unittests then changed to accomodate them (in Nova) | 01:14 |
| marun | cyeoh: maintaining them once a given api stabilizes will still be tempest's responsibility | 01:14 |
| *** mkollaro has quit IRC | 01:14 | |
| marun | cyeoh: so that backwards compatibility is still protected | 01:15 |
| cyeoh | because well people just don't think about backwards compatibility | 01:15 |
| marun | cyeoh: agree completely. | 01:15 |
| cyeoh | having the tests maintained separately in tempest acts a safeguard | 01:15 |
| marun | as per the etherpad, the suggested approach ensures that this characteristic is maintained | 01:16 |
| cyeoh | and makes people think again about what they are really doing. and stops these changes slipping through | 01:16 |
| cyeoh | oh ok I must have missed it :-) | 01:16 |
| marun | lines 25 and 29 | 01:16 |
| marun | cyeoh: the idea is that so long as an api is unstable, the test should be in-tree since it's easier to iterate and tests can be vetted pre-merge | 01:17 |
| marun | cyeoh: once an api has stabilized, backwards compatibility is required and the tests are put under tempest's control | 01:17 |
| *** gema has quit IRC | 01:17 | |
| marun | cyeoh: does that make sense? | 01:17 |
| cyeoh | marun: so its not clear to me from that if its possible for a backwards breaking change to get merged without a corresponding tempest approval for an 'experimental' API | 01:18 |
| cyeoh | I really want that not to be possible (or at least a project to be able to set that for an experimental api) | 01:19 |
| *** mkollaro has joined #openstack-qa | 01:19 | |
| marun | cyeoh: I think the thought process is more "we have a public-facing api. it's not allowed to be 'official' until tempest has tests for it" | 01:20 |
| mtreinish | cyeoh: the thing I've seen with experimental apis is that they want to do backwards breaking changes. (which is why they're marked as experimental) I've already seen 2 this cycle. | 01:20 |
| cyeoh | mtreinish: so we're perhaps talking about 2 different situations | 01:20 |
| mtreinish | I think we are | 01:21 |
| mtreinish | the tests belong in tempest if they're willing to conform the stability guidelines | 01:21 |
| cyeoh | I would really like to see in the future (once we have microversions) that *ALL* new Nova apis are experimental for one cycle | 01:21 |
| marun | cyeoh: backwards compatible and experimental do not mix | 01:21 |
| mtreinish | cyeoh: all new microversions, or new apis? | 01:21 |
| cyeoh | its what we would previously have released as "stable" but we know now we our "oops" percentage is low, but too high | 01:22 |
| cyeoh | new APIs, not necessarily each microversion, tasks api for example is a good example | 01:22 |
| cyeoh | so we're telling users that: "this is what we think our stable api is going to be, and we're not going to arbitarily change it" | 01:23 |
| marun | so are those api's truly experimental? | 01:23 |
| cyeoh | eg. we've gone through a *lot* of spec design planning etc, a lot of analysis etc | 01:23 |
| cyeoh | and we really think we've got it right. | 01:23 |
| marun | it sounds like splitting hairs on what 'experimental' means | 01:23 |
| cyeoh | but history shows us that there's a non zero percentage change we got something wrong. So we might tweak it a bit | 01:23 |
| marun | the more important part, I think, is backwards compatibility | 01:24 |
| cyeoh | but we'll lock it down in one cycle | 01:24 |
| marun | if bc is required, then tests have to be in tempest | 01:24 |
| marun | otherwise, they can evolve in-tree | 01:24 |
| marun | until bc becomes necessary | 01:24 |
| mtreinish | cyeoh: yeah I'm just looking at it from the backwards compatibility standpoint. We can make breaking api changes but they have to be backported if they cross a release boundry | 01:24 |
| *** gema has joined #openstack-qa | 01:25 | |
| mtreinish | if we have to start playing games with tagging based on time then I really am not sure we should be testing it in tempest | 01:25 |
| cyeoh | marun: so evolving in-tree I see as losing a safety net (eg a patch going going through really quickly that changes both test and implementation) | 01:25 |
| mtreinish | the tag we were talking about before I was envisioning more for tests that existed before we started being more explicit about what should be in tree | 01:25 |
| cyeoh | mtreinish: well but if we can just have a tag that says - only test against the latest version - that'd be fine with me. Gets the safety net and the tag can be removed when its no longer experimental | 01:26 |
| marun | cyeoh: you don't need iteration to be expensive until backwards compatibility is required. | 01:26 |
| cyeoh | and we would not keep experimental APIs in Nova for more than once cycle anyway | 01:26 |
| marun | cyeoh: but you're free to spend your time how you wish, of course. | 01:26 |
| cyeoh | marun: but in these cases we're not expecting to be iterating | 01:26 |
| *** adalbas has joined #openstack-qa | 01:27 | |
| marun | cyeoh: I tend to have too little, though, as opposed to too much. | 01:27 |
| mtreinish | then the normal feature flag would cover that use case if it's stable after the first cycle it's introduced | 01:27 |
| marun | cyeoh: well, then do it in tempest | 01:27 |
| *** adalbas has quit IRC | 01:27 | |
| mtreinish | you would just need to backport any fixes that made a backwards incompatible change | 01:27 |
| cyeoh | with nova-specs and CD we don't expect to be iterating - this for "oops" moments | 01:27 |
| marun | cyeoh: but being able to iterate on code and tests in-tree is valuable for neutron at least. | 01:27 |
| cyeoh | marun: oh I'm not saying that projects shouldn't be able to do what you want | 01:27 |
| marun | have to run | 01:27 |
| *** marun has quit IRC | 01:28 | |
| cyeoh | I'm just saying thats not what I want for nova :-) | 01:28 |
| mtreinish | cyeoh: so I actually don't think what you want for nova is incompatible with what I'm adding to the readme | 01:29 |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-neutron 97264,2,ab81ed2 | 01:29 |
| mtreinish | it's just that you'll have to backport a fix if we do end up slipping | 01:29 |
| mtreinish | I really just added that as a response to ironic who has tests but have planned "features" that are making backwards incompatible api changes | 01:30 |
| openstackgerrit | A change was merged to openstack/tempest: Add bug skip for test_container_synchronization() https://review.openstack.org/92606 | 01:32 |
| cyeoh | mtreinish: hrm. perhaps. I guess it depends on how we define experimental & stable. If tell nova api users that say an icehouse update may change an "experimental API", rather than it only changing between icehouse and juno | 01:32 |
| cyeoh | though I think i'd prefer that we could at least say that our experimental apis are stable within a release | 01:33 |
| mtreinish | yeah but with branchless tempest we're saying that any future breaking api changes have to be backported anyway so it's about the same | 01:33 |
| *** leej has joined #openstack-qa | 01:33 | |
| mtreinish | yeah it's just a definition thing | 01:33 |
| *** nosnos has joined #openstack-qa | 01:33 | |
| mtreinish | I don't think we really have a clear definition on any of this anyway :) | 01:34 |
| *** nati_uen_ has quit IRC | 01:36 | |
| *** yamahata has joined #openstack-qa | 01:37 | |
| cyeoh | yea and the whole "experimental for one cycle" is a new thing for Nova too anyway :-) | 01:38 |
| mtreinish | cyeoh: but add something to the meeting agenda about the experimental tag, it'll probably be good to discuss with more people | 01:41 |
| mtreinish | I may not be the biggest fan of the idea, because I think we should only gate on things that shouldn't be changing but I think it's worth discussing one more time | 01:42 |
| cyeoh | sure ok - I'll add it. I guess I see this as handling the cases where it *shouldn't* be changing but unfortunately has to because we don't want to live with broken for another 10 years. | 01:44 |
| openstackrecheck | Console logs not available after 13:23s for gate-trove-pep8 77551,18,a1766c9 | 01:45 |
| *** weshay has quit IRC | 01:47 | |
| *** leej has quit IRC | 01:55 | |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm-partial-ncpu 97598,1,2125f27 | 01:59 |
| openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds list users tests in v3 https://review.openstack.org/93704 | 01:59 |
| openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds keystone roles v2 tests in v3 https://review.openstack.org/89357 | 02:03 |
| *** mlavalle has quit IRC | 02:13 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full-icehouse 98804,2,7b12b2a | 02:14 |
| *** nati_ueno has joined #openstack-qa | 02:23 | |
| *** zhikunliu has joined #openstack-qa | 02:26 | |
| *** mkollaro has quit IRC | 02:27 | |
| openstackgerrit | A change was merged to openstack/tempest: dump console if we time out on a server op https://review.openstack.org/97812 | 02:27 |
| openstackrecheck | Console logs not available after 13:31s for check-tempest-dsvm-postgres-full 95569,4,cec612e | 02:28 |
| openstackrecheck | Console logs not available after 13:35s for check-tempest-dsvm-neutron-icehouse 98443,7,6887323 | 02:41 |
| *** bkopilov has quit IRC | 02:46 | |
| *** rwsu has quit IRC | 02:52 | |
| openstackrecheck | Console logs not available after 13:43s for check-tempest-dsvm-full 98819,3,083e58b | 02:55 |
| openstackrecheck | Console logs not available after 13:27s for gate-tempest-dsvm-full 97255,3,8066515 | 03:09 |
| *** nati_ueno has quit IRC | 03:10 | |
| *** lbragstad has quit IRC | 03:11 | |
| *** SridharG has joined #openstack-qa | 03:20 | |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-full 97944,1,c7b72d8 | 03:23 |
| *** SridharG has quit IRC | 03:35 | |
| openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm 84295,8,3fbdb6d | 03:38 |
| *** nosnos has quit IRC | 03:43 | |
| openstackrecheck | Console logs not available after 13:33s for gate-tempest-dsvm-neutron-large-ops 95844,3,2a9bd7f | 03:52 |
| *** Longgeek has joined #openstack-qa | 03:52 | |
| *** dims_ has quit IRC | 03:53 | |
| *** Madan has joined #openstack-qa | 03:54 | |
| *** coolsvap|afk is now known as coolsvap | 03:57 | |
| *** Longgeek has quit IRC | 03:57 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Mising CLI test for Keystone https://review.openstack.org/98749 | 04:00 |
| raies | afazekas: hi | 04:02 |
| raies | afazekas: my two of patches are being failed due to cell not being enabled in devstack. Can you please assist me in this to check for cell enabling in devstack ?? | 04:03 |
| raies | afazekas: https://review.openstack.org/#/c/98735/ , https://review.openstack.org/#/c/90337/ | 04:04 |
| openstackrecheck | Console logs not available after 13:23s for gate-ceilometer-python26 98834,1,d64029c | 04:06 |
| *** Mike77 has joined #openstack-qa | 04:09 | |
| *** bkopilov has joined #openstack-qa | 04:13 | |
| openstackgerrit | A change was merged to openstack/tempest: Place baremetal common methods in base class https://review.openstack.org/94436 | 04:15 |
| openstackrecheck | Console logs not available after 13:25s for gate-python-glanceclient-python26 98862,1,eda86f2 | 04:20 |
| *** yfried has quit IRC | 04:21 | |
| *** nosnos has joined #openstack-qa | 04:24 | |
| *** Abhijeet has joined #openstack-qa | 04:29 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-postgres-full-icehouse 94944,12,1fa88f5 | 04:35 |
| openstackgerrit | Ian Wienand proposed a change to openstack-dev/devstack: Expand variables in get_meta_section_files https://review.openstack.org/98951 | 04:35 |
| *** lbragstad has joined #openstack-qa | 04:38 | |
| *** k4n0 has joined #openstack-qa | 04:38 | |
| *** dims_ has joined #openstack-qa | 04:46 | |
| openstackrecheck | Console logs not available after 13:46s for check-grenade-dsvm-partial-ncpu 98854,1,da4cf5b | 04:50 |
| *** dims_ has quit IRC | 04:50 | |
| *** ildikov has quit IRC | 04:51 | |
| *** Poornima has joined #openstack-qa | 04:53 | |
| *** harlowja is now known as harlowja_away | 04:59 | |
| *** zhikunliu has quit IRC | 05:04 | |
| openstackrecheck | Console logs not available after 13:30s for check-tempest-dsvm-full-icehouse 94314,3,24af831 | 05:04 |
| *** tkammer_ has joined #openstack-qa | 05:06 | |
| *** lbragstad has quit IRC | 05:13 | |
| *** ildikov has joined #openstack-qa | 05:15 | |
| *** lbragstad has joined #openstack-qa | 05:15 | |
| *** zhikunliu has joined #openstack-qa | 05:17 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-full 85314,12,89d562e | 05:18 |
| raies | afazekas: hi | 05:19 |
| *** yfried has joined #openstack-qa | 05:24 | |
| openstackrecheck | Console logs not available after 13:31s for check-tempest-dsvm-postgres-full 95436,3,3c4b4f2 | 05:32 |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/95879 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1226412 | 05:42 |
| *** GMann has joined #openstack-qa | 05:44 | |
| *** dims_ has joined #openstack-qa | 05:47 | |
| *** Longgeek has joined #openstack-qa | 05:49 | |
| *** Longgeek has quit IRC | 05:49 | |
| *** Longgeek has joined #openstack-qa | 05:50 | |
| *** Longgeek has quit IRC | 05:51 | |
| *** Longgeek has joined #openstack-qa | 05:51 | |
| *** dims_ has quit IRC | 05:51 | |
| openstackrecheck | Console logs not available after 13:25s for gate-nova-python26 98874,1,6447fa9 | 05:55 |
| *** Manishanker has joined #openstack-qa | 05:57 | |
| raies | afazekas: you there ?? | 05:58 |
| *** Poornima has quit IRC | 06:01 | |
| *** Longgeek has quit IRC | 06:04 | |
| *** Longgeek has joined #openstack-qa | 06:04 | |
| *** Longgeek has quit IRC | 06:04 | |
| * afazekas waking up | 06:05 | |
| *** Longgeek has joined #openstack-qa | 06:05 | |
| * afazekas warn: regeneration cycle incomplete.. | 06:06 | |
| afazekas | raies: hi | 06:06 |
| *** Longgeek has quit IRC | 06:06 | |
| *** Longgeek_ has joined #openstack-qa | 06:06 | |
| *** Poornima has joined #openstack-qa | 06:08 | |
| raies | afazekas: take the case of this https://github.com/openstack/tempest/blob/master/tempest/api/network/test_fwaas_extensions.py#L51 | 06:08 |
| raies | afazekas: if fwaas is not enabled then these tests should skip | 06:08 |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full-icehouse 95437,4,c31e719 | 06:09 |
| raies | afazekas: but when I run it with fwaas not enabled then errror is traced saying resource not found | 06:09 |
| raies | afazekas: can you please check ?? | 06:09 |
| afazekas | traceback ? | 06:10 |
| raies | hold I am giving it in pastebit | 06:12 |
| *** Poornima has quit IRC | 06:13 | |
| raies | afazekas: please check it pastebin.com/egfiN1C7 | 06:13 |
| *** Poornima has joined #openstack-qa | 06:13 | |
| *** Poornima has quit IRC | 06:16 | |
| *** Poornima has joined #openstack-qa | 06:16 | |
| *** dims_ has joined #openstack-qa | 06:17 | |
| afazekas | tempest.conf [network-feature-enabled]api_extensions ?? | 06:17 |
| raies | afazekas: have you seen ?? | 06:17 |
| raies | afazekas: "api_extensions = all" | 06:18 |
| raies | it is in tempest.conf | 06:18 |
| afazekas | tempest.conf [network-feature-enabled]api_extensions needs to be something else than all, and it should not contain the 'fwaas' string | 06:19 |
| raies | and q-fwaas is not enabled in localrc | 06:19 |
| raies | something else I didn't get | 06:19 |
| *** Poornima has quit IRC | 06:20 | |
| *** mkoderer has joined #openstack-qa | 06:21 | |
| raies | afazekas: tempest.conf is generated, how can I change it to something else. and what to change in api_extensions | 06:21 |
| *** dims_ has quit IRC | 06:22 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full 98525,1,bf4fe30 | 06:22 |
| raies | afazekas: am I making sense ?? | 06:24 |
| afazekas | raies: you need to enumerate all extensions excluding fwaas | 06:24 |
| raies | how can I list it out in tempest.conf ?? it se auto generated right | 06:26 |
| *** Mike77 has quit IRC | 06:28 | |
| raies | afazekas: can you please show that me in an example ?? | 06:29 |
| *** evgenyf has joined #openstack-qa | 06:30 | |
| raies | afazekas: you there ?? | 06:32 |
| raies | afazekas: got that :) only one thing I need to clarify, for this purpose I think changes will be in tempest.config file to change default['all'] to 'xyz' what you say ?? | 06:36 |
| *** yrabl has joined #openstack-qa | 06:36 | |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm 94775,5,9ff98a4 | 06:38 |
| afazekas | git grep is_extension_enabled |grep network | grep -v fwaas |cut -f 2 -d\' | paste -sd "," | 06:40 |
| afazekas | agent,dhcp_agent_scheduler,l3_agent_scheduler,lbaas_agent_scheduler,lbaas,quotas,allowed-address-pairs,network,network,extra_dhcp_opt,router,lbaas,metering,router,router,security-group,security-group,service-type,vpnaas,lbaas,network | 06:40 |
| afazekas | man paste | 06:41 |
| openstackgerrit | A change was merged to openstack/tempest: Consolidate _check_public_network_connectivity into NetworkScenarioTest https://review.openstack.org/95831 | 06:42 |
| openstackgerrit | A change was merged to openstack/tempest: Consolidate _check_tenant_network_connectivity in NetworkScenarioTest https://review.openstack.org/95844 | 06:42 |
| *** lbragstad has quit IRC | 06:49 | |
| *** luqas has joined #openstack-qa | 06:50 | |
| *** lbragstad has joined #openstack-qa | 06:50 | |
| * afazekas I will return | 06:50 | |
| *** afazekas is now known as __afazekas | 06:51 | |
| openstackrecheck | Console logs not available after 13:30s for check-grenade-dsvm-partial-ncpu 98871,1,0a09026 | 06:51 |
| *** salv-orlando has quit IRC | 06:58 | |
| *** nosnos has quit IRC | 06:59 | |
| *** matsuhashi has quit IRC | 07:01 | |
| *** matsuhas_ has joined #openstack-qa | 07:01 | |
| *** mkollaro has joined #openstack-qa | 07:02 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Compute "cells" basic API test https://review.openstack.org/90337 | 07:04 |
| *** nosnos has joined #openstack-qa | 07:05 | |
| openstackrecheck | Console logs not available after 13:30s for gate-tempest-python27 98868,2,7c376bd | 07:06 |
| *** jpich has joined #openstack-qa | 07:06 | |
| raies | __afazekas: hi do you know how can I enable service required to run command "neutron cisco-network-profile-list" ?? | 07:12 |
| raies | and how to enable it in devstack ?? | 07:12 |
| *** leseb has joined #openstack-qa | 07:16 | |
| *** dims_ has joined #openstack-qa | 07:17 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-postgres-full 95278,3,5453c57 | 07:19 |
| *** dims_ has quit IRC | 07:22 | |
| *** bauzas has joined #openstack-qa | 07:26 | |
| *** andreaf has quit IRC | 07:31 | |
| openstackrecheck | Console logs not available after 13:27s for gate-tempest-dsvm-postgres-full 96955,9,6655467 | 07:33 |
| *** mwagner_lap has quit IRC | 07:34 | |
| *** mwagner_lap has joined #openstack-qa | 07:36 | |
| *** jlibosva has joined #openstack-qa | 07:42 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 98693,1,af37ab6 | 07:47 |
| *** thomasbiege has joined #openstack-qa | 07:47 | |
| *** ajo|mac has joined #openstack-qa | 07:50 | |
| *** SridharG has joined #openstack-qa | 07:51 | |
| *** afazekas has joined #openstack-qa | 07:55 | |
| *** vponomaryov has joined #openstack-qa | 07:55 | |
| *** andreaf has joined #openstack-qa | 07:58 | |
| openstackrecheck | Console logs not available after 13:38s for check-tempest-dsvm-full 95279,4,f5459d9 | 08:00 |
| *** gszasz has joined #openstack-qa | 08:02 | |
| *** vrovachev has joined #openstack-qa | 08:05 | |
| *** bauzas has quit IRC | 08:07 | |
| openstackrecheck | Console logs not available after 13:27s for gate-nova-docs 94915,9,0227830 | 08:14 |
| *** jordanP has joined #openstack-qa | 08:14 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Correcting 'id' type in compute schema https://review.openstack.org/98971 | 08:14 |
| *** qba73 has joined #openstack-qa | 08:15 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Correcting 'id' type in compute schema https://review.openstack.org/98971 | 08:17 |
| *** dims_ has joined #openstack-qa | 08:18 | |
| *** leseb has quit IRC | 08:18 | |
| *** safchain has joined #openstack-qa | 08:20 | |
| *** dims_ has quit IRC | 08:22 | |
| *** dkorn has joined #openstack-qa | 08:24 | |
| openstackrecheck | Console logs not available after 13:41s for gate-trove-tox-checkdeletions 98902,1,4bd7976 | 08:30 |
| *** ajo|mac has quit IRC | 08:31 | |
| *** ajo|mac has joined #openstack-qa | 08:33 | |
| *** dkorn has quit IRC | 08:35 | |
| *** matsuhas_ has quit IRC | 08:38 | |
| *** nosnos has quit IRC | 08:40 | |
| *** mkollaro has quit IRC | 08:40 | |
| *** lbragstad has quit IRC | 08:43 | |
| openstackrecheck | Console logs not available after 13:28s for check-tempest-dsvm-postgres-full 98894,1,7ddbb37 | 08:43 |
| *** nosnos has joined #openstack-qa | 08:43 | |
| *** lbragstad has joined #openstack-qa | 08:43 | |
| *** nosnos has quit IRC | 08:44 | |
| *** matsuhashi has joined #openstack-qa | 08:44 | |
| *** tkammer_ has quit IRC | 08:50 | |
| *** jhenner has joined #openstack-qa | 08:50 | |
| *** leseb has joined #openstack-qa | 08:50 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Missing Neutron CLI tests https://review.openstack.org/98974 | 08:54 |
| *** mmaglana has quit IRC | 08:54 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: An example of Certificate Generation with devstack https://review.openstack.org/47526 | 08:55 |
| *** giulivo has joined #openstack-qa | 08:56 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: SSL protected cinder endpoints https://review.openstack.org/97632 | 08:57 |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 98691,2,cf02f05 | 08:58 |
| *** Abhijeet has quit IRC | 08:59 | |
| *** thomasbiege1 has joined #openstack-qa | 09:00 | |
| *** nosnos has joined #openstack-qa | 09:03 | |
| *** leseb has quit IRC | 09:03 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: Use identity_uri instead of auth fragments https://review.openstack.org/94545 | 09:03 |
| *** thomasbiege has quit IRC | 09:04 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: Use identity_uri instead of auth fragments https://review.openstack.org/94545 | 09:05 |
| *** tkammer_ has joined #openstack-qa | 09:09 | |
| *** zhikunliu has quit IRC | 09:09 | |
| *** leseb has joined #openstack-qa | 09:14 | |
| openstackrecheck | Console logs not available after 13:25s for gate-nova-pep8 98865,2,4a52c87 | 09:16 |
| *** dims_ has joined #openstack-qa | 09:19 | |
| *** luqas has quit IRC | 09:20 | |
| *** evgenyf has quit IRC | 09:21 | |
| *** dims_ has quit IRC | 09:25 | |
| openstackrecheck | Console logs not available after 13:32s for gate-nova-docs 98864,2,193a1dc | 09:30 |
| *** gszasz has quit IRC | 09:38 | |
| *** dmorita has quit IRC | 09:41 | |
| *** weshay has joined #openstack-qa | 09:43 | |
| mkoderer | oomichi: ping | 09:46 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: VPNaas IPSec policies tests https://review.openstack.org/47816 | 09:46 |
| *** luqas has joined #openstack-qa | 09:49 | |
| *** dkorn has joined #openstack-qa | 09:58 | |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 98885,1,3d4b6cb | 09:59 |
| *** salv-orlando has joined #openstack-qa | 10:00 | |
| *** leseb has quit IRC | 10:09 | |
| *** leseb has joined #openstack-qa | 10:10 | |
| *** leseb has quit IRC | 10:12 | |
| openstackrecheck | Console logs not available after 13:28s for check-tempest-dsvm-full 98443,7,15bfb2a | 10:14 |
| *** evgenyf has joined #openstack-qa | 10:16 | |
| *** dkorn has quit IRC | 10:19 | |
| *** dims_ has joined #openstack-qa | 10:21 | |
| openstackgerrit | Pranali Deore proposed a change to openstack/tempest: Added test case for nova keypair https://review.openstack.org/79549 | 10:22 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/96829 | 10:23 |
| *** dims_ has quit IRC | 10:27 | |
| openstackrecheck | Console logs not available after 13:28s for gate-python-keystoneclient-python26 96265,4,3090321 | 10:27 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: make logformat match oslo https://review.openstack.org/98819 | 10:35 |
| *** ylobankov has joined #openstack-qa | 10:36 | |
| openstackrecheck | Console logs not available after 13:33s for gate-requirements-integration-dsvm 93156,6,6849d22 | 10:42 |
| *** ajo|mac has quit IRC | 10:43 | |
| *** mmaglana has joined #openstack-qa | 10:55 | |
| openstackrecheck | Console logs not available after 13:34s for check-tempest-dsvm-neutron 98672,1,77f0990 | 10:56 |
| *** dkorn has joined #openstack-qa | 10:56 | |
| *** dkorn has quit IRC | 11:00 | |
| *** mmaglana has quit IRC | 11:00 | |
| *** coolsvap is now known as coolsvap|afk | 11:04 | |
| *** cyeoh has quit IRC | 11:06 | |
| *** jordanP has quit IRC | 11:08 | |
| *** gszasz has joined #openstack-qa | 11:08 | |
| *** cyeoh has joined #openstack-qa | 11:10 | |
| openstackrecheck | Console logs not available after 13:24s for gate-heat-pep8 98687,2,4fcc24e | 11:12 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Missing Swift CLI tests https://review.openstack.org/99005 | 11:16 |
| *** luqas has quit IRC | 11:22 | |
| *** dims_ has joined #openstack-qa | 11:23 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Missing Neutron CLI tests https://review.openstack.org/98974 | 11:23 |
| openstackrecheck | Console logs not available after 13:29s for gate-heat-python26 98686,2,0bde3d0 | 11:27 |
| *** salv-orlando has quit IRC | 11:27 | |
| *** salv-orlando has joined #openstack-qa | 11:32 | |
| *** leseb has joined #openstack-qa | 11:32 | |
| *** yamahata has quit IRC | 11:32 | |
| *** yfried has quit IRC | 11:40 | |
| openstackrecheck | Console logs not available after 13:31s for gate-tempest-python27 98899,1,ca6fe7f | 11:42 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: don't update apt repos https://review.openstack.org/99016 | 11:45 |
| *** GMann has quit IRC | 11:45 | |
| *** mkollaro has joined #openstack-qa | 11:54 | |
| openstackrecheck | Console logs not available after 13:28s for gate-heat-pep8 98683,2,9f14e44 | 11:55 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Enable horizon on Fedoras https://review.openstack.org/97532 | 11:56 |
| openstackgerrit | afazekas proposed a change to openstack-infra/elastic-recheck: Add query for 1328500 https://review.openstack.org/99020 | 11:57 |
| *** gszasz has quit IRC | 11:58 | |
| *** tkammer_ has quit IRC | 12:00 | |
| *** mkollaro1 has joined #openstack-qa | 12:00 | |
| *** mkollaro has quit IRC | 12:00 | |
| *** jhenner has quit IRC | 12:01 | |
| *** jhenner has joined #openstack-qa | 12:02 | |
| *** tkammer_ has joined #openstack-qa | 12:03 | |
| *** adalbas has joined #openstack-qa | 12:04 | |
| *** afazekas has quit IRC | 12:09 | |
| *** aysyd has joined #openstack-qa | 12:09 | |
| *** yfried has joined #openstack-qa | 12:10 | |
| openstackrecheck | Console logs not available after 13:41s for gate-heat-python27 97976,3,d9c36d4 | 12:11 |
| *** jordanP has joined #openstack-qa | 12:13 | |
| *** roeyc has joined #openstack-qa | 12:13 | |
| mkollaro1 | are there any tests that try if nova-scheduler works properly when a node fails during VM creation? say, when a network interface goes down | 12:13 |
| *** jhenner has quit IRC | 12:18 | |
| *** jhenner has joined #openstack-qa | 12:20 | |
| *** gszasz has joined #openstack-qa | 12:23 | |
| *** gszasz has quit IRC | 12:24 | |
| *** afazekas has joined #openstack-qa | 12:24 | |
| *** dkranz has joined #openstack-qa | 12:24 | |
| *** Longgeek_ has quit IRC | 12:25 | |
| *** Longgeek has joined #openstack-qa | 12:26 | |
| *** mwagner_lap has quit IRC | 12:26 | |
| openstackrecheck | Console logs not available after 13:39s for gate-heat-python27 97981,3,f8f0bfb | 12:26 |
| *** dhellmann has quit IRC | 12:28 | |
| *** oomichi_ has joined #openstack-qa | 12:31 | |
| *** Madan has quit IRC | 12:32 | |
| *** Ju has quit IRC | 12:35 | |
| openstackgerrit | Marc Koderer proposed a change to openstack/tempest: Add decorator for safe tearDown https://review.openstack.org/84645 | 12:41 |
| openstackrecheck | Console logs not available after 13:31s for check-tempest-dsvm-full 97980,3,92211c1 | 12:41 |
| *** lbragstad has quit IRC | 12:45 | |
| *** SridharG has quit IRC | 12:46 | |
| *** gszasz has joined #openstack-qa | 12:47 | |
| *** luqas has joined #openstack-qa | 12:48 | |
| *** SridharG has joined #openstack-qa | 12:48 | |
| *** dkranz has quit IRC | 12:48 | |
| *** ekhugen has joined #openstack-qa | 12:49 | |
| *** GMann has joined #openstack-qa | 12:50 | |
| *** Manishanker has quit IRC | 12:51 | |
| *** gszasz has quit IRC | 12:52 | |
| *** mkollaro has joined #openstack-qa | 12:52 | |
| *** afazekas has quit IRC | 12:53 | |
| *** SridharGaddam has joined #openstack-qa | 12:55 | |
| *** SridharG has quit IRC | 12:55 | |
| openstackrecheck | Console logs not available after 13:25s for gate-heat-python26 97982,3,14b44b5 | 12:56 |
| *** mkollaro1 has quit IRC | 12:56 | |
| *** mmaglana has joined #openstack-qa | 12:56 | |
| *** GMann has quit IRC | 12:56 | |
| *** gszasz has joined #openstack-qa | 12:57 | |
| *** dims_ has quit IRC | 12:59 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable E129,E128 rules https://review.openstack.org/98909 | 12:59 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Bump hacking to 0.9.x series https://review.openstack.org/98868 | 12:59 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable E113,E111,H302,F812,E713 rules https://review.openstack.org/98899 | 12:59 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H407,H305,E123,H307,E122 rules https://review.openstack.org/98918 | 12:59 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H402,E251,E265 rules https://review.openstack.org/98903 | 12:59 |
| *** bkopilov has quit IRC | 12:59 | |
| sdague | mtreinish: you do realize you are going to be rebasing those forever :) | 13:01 |
| *** mmaglana has quit IRC | 13:01 | |
| mtreinish | sdague: probably, so once the requirements bump lands I'll be bugging you and dkranz to push them through fast | 13:02 |
| sdague | yeh, I'm not +2ing any of those | 13:02 |
| sdague | I'm pretty philosophically opposed to landing hacking changes at this point | 13:03 |
| *** jhenner has quit IRC | 13:03 | |
| sdague | it's giant churn at a point when we have real things to do | 13:03 |
| sdague | it's also consuming 10% of all our test resources every time that patch series gets rebased | 13:05 |
| *** yamahata has joined #openstack-qa | 13:05 | |
| *** gszasz has quit IRC | 13:05 | |
| *** tkammer_ has quit IRC | 13:05 | |
| *** k4n0 has quit IRC | 13:06 | |
| mkoderer | sdague: dkranz commented his qa_spec: https://review.openstack.org/#/c/93037 | 13:06 |
| *** gszasz has joined #openstack-qa | 13:06 | |
| mkoderer | He might add this aspect to the spec since we both misunderstand it :) | 13:06 |
| *** amotoki_ has joined #openstack-qa | 13:06 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Use correct base test class for test_stress unit tests https://review.openstack.org/99046 | 13:07 |
| *** afazekas has joined #openstack-qa | 13:07 | |
| mtreinish | sdague: ^^^ that should fix the issue you saw with you log formatter patch | 13:07 |
| sdague | mtreinish: actually, I just decoupled it | 13:08 |
| sdague | mtreinish: https://review.openstack.org/#/c/98819/ | 13:08 |
| mkoderer | mtreinish: ohh nice... seems I missed that one | 13:08 |
| *** dustins has joined #openstack-qa | 13:08 | |
| mtreinish | sdague: yeah but the only reason you were seeing an issue was because the unit tests shouldn't have been using that base class | 13:09 |
| mkoderer | mtreinish: I wait for jenkins and you got my +A | 13:10 |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: [Fedora] Devstack fails to install in firewalld less system https://review.openstack.org/99047 | 13:10 |
| sdague | mtreinish: sure. I'd still like to land the passing patch. As that could get through the gate by this afternoon. | 13:11 |
| sdague | and we can recouple to conf later if we think it's importantly | 13:11 |
| sdague | important | 13:11 |
| mtreinish | sdague: I think it's importantly important, because if we change the default log format for any reason the subunit logs would be incorrect again | 13:13 |
| sdague | that's in log.py, which is in oslo, and basically been static since the end of time | 13:13 |
| sdague | so the odds of changing that format are roughly the same odds as rewriting tempest in go | 13:14 |
| afazekas | ianw, sdague: https://review.openstack.org/#/c/99047/1 | 13:14 |
| mtreinish | sdague: yeh, but with the rate you're changing logging how soon until you reset the default like you did with the default_log_level stuff :) | 13:15 |
| sdague | mtreinish: then we'll patch it then :) | 13:15 |
| sdague | if we keep not making incremental progress because "it could be better", we make no progress | 13:15 |
| mtreinish | and it's also for consitency for people who set there own log format in conf. I think having it be consistent in both places is important | 13:15 |
| sdague | mtreinish: if you find me a person who does that, I'll agree | 13:16 |
| sdague | because no one does that | 13:16 |
| *** mriedem has joined #openstack-qa | 13:16 | |
| mtreinish | we can't assume that, it's a valid config option. That's like the people who want to make breaking api changes because they claim no one uses the api | 13:17 |
| *** jhenner has joined #openstack-qa | 13:17 | |
| sdague | mtreinish: it doesn't make it any worse | 13:17 |
| *** lbragstad has joined #openstack-qa | 13:17 | |
| sdague | this is the thing that drives me nuts. The patch there is a demonstrable improvement that could make it much easier to see what's going on | 13:18 |
| sdague | it doesn't regress anything | 13:18 |
| * afazekas http://www.youtube.com/watch?v=GPMpMroOkwc | 13:18 | |
| sdague | and yet we'll spend 4 days in round trips to make that perfect, instead of land "better", and be able to make headway on additional things | 13:18 |
| mtreinish | sdague: that's not what this is. The previous rev was the correct way to do it, you just hit a bug. Instead of fixing the underlying problem you worked around it | 13:19 |
| *** leseb has quit IRC | 13:19 | |
| sdague | and so we've got a patch now, which works fine, and once the other patch lands we can circle around again | 13:20 |
| *** leseb has joined #openstack-qa | 13:20 | |
| *** leseb has quit IRC | 13:24 | |
| *** gszasz has quit IRC | 13:24 | |
| vrovachev | sdague: hello :) I answered you: https://review.openstack.org/#/c/64136/ | 13:25 |
| sdague | vrovachev: thanks | 13:25 |
| *** lbragstad1 has joined #openstack-qa | 13:26 | |
| vrovachev | sdague: thanks you :) | 13:26 |
| sdague | vrovachev: so in generally, I actually want to completely remove the ceilometer polling from the gate runs | 13:26 |
| *** gszasz has joined #openstack-qa | 13:27 | |
| sdague | because I think it's generating a lot of extra load, and it's not going to be useful for testing | 13:27 |
| afazekas | sdague: what is your opinion about doubling the build timeout (may be the build interval as well)? | 13:28 |
| *** lbragstad has quit IRC | 13:28 | |
| vrovachev | sdague: yes, of course. We have not yet added polling testing from gate | 13:28 |
| sdague | afazekas: do you have any evidence that resources would actually build in the increased timeout | 13:29 |
| sdague | because it's already pretty high | 13:29 |
| *** lbragstad has joined #openstack-qa | 13:29 | |
| afazekas | sdague: yes it is pretty high, I do not have any strong evidence | 13:30 |
| sdague | afazekas: I think some evidence that it would help would be good | 13:30 |
| openstackrecheck | Console logs not available after 13:56s for gate-python-keystoneclient-python33 79542,9,5abcece | 13:30 |
| afazekas | sdague: but IMHO it wort to try | 13:30 |
| sdague | because I don't think it would | 13:30 |
| *** lbragstad1 has quit IRC | 13:30 | |
| sdague | afazekas: no, it's randomly shooting in the dark | 13:30 |
| *** dims_ has joined #openstack-qa | 13:30 | |
| sdague | because we don't know why systems don't build | 13:31 |
| sdague | so the energy should be spent on increasing the understanding there | 13:31 |
| afazekas | https://bugs.launchpad.net/tempest/+bug/1316926 here in the first 2 case it might help, in 3th probbaly not | 13:31 |
| sdague | afazekas: no, I don't think it would | 13:32 |
| *** Obulpathi has joined #openstack-qa | 13:32 | |
| afazekas | sdague: what is your opinion about having the services to print all threads stack trace on timeout ? | 13:33 |
| DinaBelova | sdague, vrovachev - about the polling in the ceilometer | 13:33 |
| sdague | what services? | 13:33 |
| sdague | afazekas: how would that help? | 13:33 |
| afazekas | sdague: n-cpu for example | 13:33 |
| DinaBelova | sdague, although currently it's not used for testing, it is still a big part of ceilometer | 13:34 |
| DinaBelova | sdague and I think we have one change on review | 13:34 |
| sdague | DinaBelova: I know, but it's really problematic | 13:34 |
| *** jgrimm has joined #openstack-qa | 13:34 | |
| DinaBelova | sdague, yeah, I guess so.. | 13:34 |
| *** Obulpathi has quit IRC | 13:34 | |
| DinaBelova | sdague, probably we may find some other solution here | 13:35 |
| sdague | and a huge part of the performance problems - http://openstack-in-production.blogspot.com/2014/03/cern-cloud-architecture-update-for.html | 13:35 |
| *** Obulpathi has joined #openstack-qa | 13:35 | |
| DinaBelova | sdague, thanks for the link, really | 13:35 |
| *** salv-orlando has quit IRC | 13:35 | |
| DinaBelova | sdague, as for the performance problems - we're working on it :) | 13:35 |
| * afazekas one thread dumper: http://www.fpaste.org/108641/ | 13:35 | |
| DinaBelova | sdague, as for the polling - probably we may poll not all the meters | 13:36 |
| *** salv-orlando has joined #openstack-qa | 13:36 | |
| DinaBelova | but only some of them :-\ | 13:36 |
| DinaBelova | although in this case it'll be not clear what we're testing really | 13:36 |
| sdague | DinaBelova: so why isn't the effort being put into creating the right notifications so you don't have to poll? | 13:36 |
| sdague | I guess that's where my confusion lies | 13:36 |
| DinaBelova | sdague, the thing is that as you know there are pollsters (that get the info from services API where it's possible) and agent, sending the notifications | 13:37 |
| * afazekas another way https://wiki.openstack.org/wiki/GuruMeditationReport | 13:37 | |
| DinaBelova | basically the thing is that lots of info is clearly collected by asking the appropriate API | 13:37 |
| DinaBelova | as services already know about these values | 13:37 |
| sdague | DinaBelova: right, but polling puts more load on a lot of systems | 13:38 |
| sdague | where as if you consumed the notifications, you wouldn't add more load | 13:38 |
| DinaBelova | sdague, but it's really a bad idea create some other service that'll know the same things that nova does already. Or it'll need to be implemented some periodic task in nova itself that'll publish this info | 13:39 |
| DinaBelova | I'm not sure why we're where we are now (as I'm still newby here) | 13:39 |
| mriedem | got a head scratcher | 13:39 |
| *** matsuhashi has quit IRC | 13:39 | |
| DinaBelova | but I guess the last variant was met cold :) | 13:40 |
| sdague | DinaBelova: nova already has a periodic thread that sends info to the scheduler | 13:40 |
| sdague | :) | 13:40 |
| *** wingwj has quit IRC | 13:40 | |
| DinaBelova | sdague, please let's wait eglynn - he's on the call now, but he may help us to understand thse moments :) | 13:40 |
| sdague | sure | 13:41 |
| mriedem | sdague: have you seen this before? http://logs.openstack.org/73/97573/5/check/check-tempest-dsvm-full/8c8f440/console.html#_2014-06-07_20_42_27_560 | 13:41 |
| mriedem | logstash is ~30% success when that shows up | 13:41 |
| DinaBelova | sdague. because as said, in current view of how does the ceilo work? pollsters tests are really useful thing :) | 13:41 |
| sdague | mriedem: that's the timeout | 13:41 |
| *** Longgeek has quit IRC | 13:42 | |
| mriedem | sdague: known bug? | 13:42 |
| *** Longgeek has joined #openstack-qa | 13:42 | |
| mriedem | DinaBelova: using ceilometer as a model for good performance? | 13:43 |
| DinaBelova | mriedem, sorry? | 13:43 |
| mriedem | "because as said, in current view of how does the ceilo work? pollsters tests are really useful thing" | 13:43 |
| mriedem | i'd hope that any new service isn't going to follow a model of polling like ceilometer | 13:43 |
| DinaBelova | I just say that current ceilometer architecture requires polling :) | 13:44 |
| DinaBelova | and if so, it's a good idea to know is it working or not | 13:44 |
| mriedem | isn't a lot of ceilometer supposed to be refactored in juno? | 13:44 |
| openstackrecheck | Console logs not available after 13:27s for gate-python-keystoneclient-python27 95678,2,3b6c8db | 13:44 |
| DinaBelova | mriedem, yes, lots of things | 13:44 |
| sdague | mriedem: the real question is why the run took 3 hrs | 13:44 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/97724 failed because of: gate-tempest-dsvm-large-ops: https://bugs.launchpad.net/bugs/1257626 | 13:44 |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/98824 failed gate-requirements-integration-dsvm in the gate queue with an unrecognized error | 13:45 |
| sdague | that timeout is there to trigger artifact generation before jenkins kills us | 13:45 |
| mriedem | ah | 13:45 |
| sdague | because if jenkins kills us, we don't get any logs | 13:46 |
| mriedem | right | 13:46 |
| *** lbragstad has quit IRC | 13:48 | |
| mriedem | looks like a 4 CPU VM | 13:48 |
| mriedem | timed out after 2 hours | 13:48 |
| mriedem | doesn't look like when this happens we don't get the pretty worker balance dump | 13:48 |
| *** lbragstad has joined #openstack-qa | 13:48 | |
| *** nosnos has quit IRC | 13:50 | |
| mriedem | sdague: aha, seeing some rpc errors in n-cpu logs | 13:53 |
| sdague | network related? | 13:53 |
| mriedem | doesn't look like it | 13:54 |
| sdague | mriedem: it was an 8 CPU vm, there were 4 workers | 13:54 |
| mtreinish | mriedem: if you want the pretty worker balance dump you can download the subunit file from logs and pipe it into the trace script | 13:56 |
| mriedem | where do i tell how many CPUs were on the VM? | 13:56 |
| *** lbragstad1 has joined #openstack-qa | 13:57 | |
| sdague | mriedem: you can tell by the number of testr workers | 13:58 |
| sdague | which will be cpu / 2 | 13:58 |
| *** lbragstad has quit IRC | 13:58 | |
| mriedem | ah ok | 13:58 |
| mriedem | figured it was 1:1 with available CPUs | 13:58 |
| openstackrecheck | Console logs not available after 13:28s for gate-glance-python27 88719,15,ed25488 | 13:58 |
| sdague | mriedem: no, that caused too many failures | 13:58 |
| mtreinish | sdague: I thought you had a patch that printed proc cpu somewhere | 13:59 |
| sdague | mtreinish: it's stuck somewhere | 13:59 |
| *** SridharGaddam is now known as SridharG | 13:59 | |
| sdague | somewhere in my 27 outstanding patches :P | 13:59 |
| sdague | or maybe it fell off | 14:00 |
| mtreinish | sdague: yeah looks like you abandonded it: https://review.openstack.org/#/c/97965/ | 14:00 |
| *** eglynn has joined #openstack-qa | 14:01 | |
| sdague | yeh, honestly, with so many outstanding patches, it's hard to remember what's in the queue for improvements | 14:02 |
| sdague | which is kind of why I'm antsy to actually get the ones that pass which make incremental improvements landed | 14:02 |
| mtreinish | sdague: so it actually looks like your log format one doesn't work correctly: http://logs.openstack.org/19/98819/5/check/check-tempest-dsvm-neutron-full-icehouse/95a1375/console.html | 14:03 |
| dims_ | sdague, seeing something weird in one of the ssh timeout logs. seems like ntpd is listening on the same ip that was supposed to be on a vm - http://paste.openstack.org/show/83523/ | 14:04 |
| dims_ | sdague, "ntpd[2114]: Listen normally on 368 br100 172.24.4.1 UDP 123" | 14:04 |
| sdague | mtreinish: ok, that's fun | 14:04 |
| sdague | well that definitely means we can't use the oslo config part | 14:05 |
| mtreinish | heh, I guess so | 14:05 |
| *** Ju_ has joined #openstack-qa | 14:06 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/elastic-recheck: Prettified all HTML files https://review.openstack.org/98445 | 14:07 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Official client based auth https://review.openstack.org/82911 | 14:07 |
| dims_ | sdague, isnt that weird? (see above) | 14:08 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: make logformat more similar to oslo https://review.openstack.org/98819 | 14:09 |
| sdague | dims_: why? | 14:10 |
| sdague | I assume ntpd is binding to 0.0.0.0 | 14:10 |
| dims_ | floating ip is for the vm right? ntpd is on the host no? | 14:10 |
| sdague | floating ip is created on the host then SNAT to the guest | 14:11 |
| sdague | on the compute | 14:11 |
| *** rbak has joined #openstack-qa | 14:11 | |
| dims_ | ah. thanks sdague | 14:11 |
| openstackrecheck | Console logs not available after 13:30s for gate-python-keystoneclient-python27 86237,2,e0c29c2 | 14:12 |
| mtreinish | sdague: on the latest rev, will we get the request id anywhere? I thought that one of big motivators for this was that we lost the req id in the console output | 14:15 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/98820 failed because of: gate-ceilometer-python26: https://bugs.launchpad.net/bugs/1327344 | 14:15 |
| *** oomichi_ has quit IRC | 14:16 | |
| sdague | mtreinish: we won't | 14:16 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/94755 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1254890 | 14:16 |
| sdague | we will get level though | 14:16 |
| *** lbragstad1 has quit IRC | 14:16 | |
| sdague | which would be an improvement all by itself | 14:16 |
| sdague | I left the commit comment, in order to get request_id we'll need a custom fixture | 14:17 |
| *** dhellmann has joined #openstack-qa | 14:20 | |
| *** leseb has joined #openstack-qa | 14:20 | |
| openstackgerrit | Rob Crittenden proposed a change to openstack-dev/devstack: Configure endpoints to use SSL natively or via proxy https://review.openstack.org/98854 | 14:22 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 14:23 |
| *** mwagner_lap has joined #openstack-qa | 14:25 | |
| *** leseb has quit IRC | 14:25 | |
| eglynn | mriedem: I missed your ceilometer question to DinaBelova earlier (... on a call) | 14:27 |
| eglynn | mriedem: i.e. "isn't a lot of ceilometer supposed to be refactored in juno?" | 14:27 |
| eglynn | mriedem: the focus for juno refactoring is around the metric storage and API end of the pipeline | 14:27 |
| eglynn | mriedem: ... but the data acquistion approach (i.e. pollsters & notification handling) will remain largely the same for Juno | 14:28 |
| *** dkranz has joined #openstack-qa | 14:28 | |
| DinaBelova | sdague, as eglynn is here we may continue that pollsters discussion :) | 14:29 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: make config generator more robust https://review.openstack.org/95770 | 14:29 |
| openstackrecheck | Console logs not available after 13:34s for check-tempest-dsvm-full 98513,7,af4a263 | 14:30 |
| * eglynn reads back thru' backlog | 14:31 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: Fix Apache site config handling on Fedora https://review.openstack.org/98537 | 14:31 |
| *** leseb has joined #openstack-qa | 14:33 | |
| *** thomasbiege1 has quit IRC | 14:34 | |
| *** thomasbiege has joined #openstack-qa | 14:35 | |
| sdague | mriedem: you have thoughts about safe substr on binary data? | 14:35 |
| sdague | mriedem: https://review.openstack.org/#/c/98505/ turns out slices aren't any good at that | 14:36 |
| *** jaypipes has joined #openstack-qa | 14:37 | |
| *** ChanServ sets mode: +o jaypipes | 14:37 | |
| mriedem | sdague: me or mtreinish? | 14:37 |
| *** kgriffs|afk is now known as kgriffs | 14:38 | |
| sdague | well, I was pinging mtreinish a lot, I thought you might have that one off the top of your head :) | 14:38 |
| mriedem | ha | 14:38 |
| *** leseb has quit IRC | 14:38 | |
| mriedem | fixing this unicode issue atm | 14:38 |
| DinaBelova | eglynn, the thing is that polling is quite expensive for cloud (and its testing), so there is change that'll turn polling of for the tests.. and that means we'll have poor opportunity for the scenario tests... | 14:38 |
| sdague | http://logs.openstack.org/05/98505/4/check/check-tempest-dsvm-full/0681279/console.html#_2014-06-09_13_44_59_185 | 14:38 |
| sdague | DinaBelova: well we're not turning polling off yet | 14:39 |
| eglynn | sdague: WRT "I actually want to completely remove the ceilometer polling from the gate runs" | 14:39 |
| *** jaypipes has quit IRC | 14:39 | |
| sdague | but I'd really like to get us there. We are going to revert back to ceilometer defaults of 600s | 14:39 |
| *** jaypipes has joined #openstack-qa | 14:39 | |
| *** ChanServ sets mode: +o jaypipes | 14:39 | |
| *** jgrimm has quit IRC | 14:40 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/96829 | 14:40 |
| DinaBelova | eglynn, the thing is that I'm quite confused... Polling is a big part of ceilo now | 14:40 |
| eglynn | sdague: ... without *any* polling, there's only a limited opportunity to make ceilometer assertions in the gate | 14:40 |
| *** jaypipes has quit IRC | 14:41 | |
| eglynn | sdague: ... but you're proposing just to revert to the out-of-the-box polling cadence, right? | 14:41 |
| sdague | eglynn: right now, yes | 14:41 |
| eglynn | DinaBelova: ... so slower polling, not no polling | 14:41 |
| *** jaypipes has joined #openstack-qa | 14:41 | |
| *** ChanServ sets mode: +o jaypipes | 14:41 | |
| *** leseb has joined #openstack-qa | 14:41 | |
| sdague | however, I do really want to get to a 0 polling model if we can | 14:41 |
| DinaBelova | eglynn, well, I was confused with the "I actually want to completely remove the ceilometer polling from the gate runs" | 14:42 |
| DinaBelova | :) | 14:42 |
| sdague | DinaBelova: "I want" :) | 14:42 |
| sdague | eglynn: so here's the rub, we actually had to build a work around in our log processing because ceilometer's polling on swift is so frequent and so verbose that it's actually materially responsible to the fact that logstash can't keep up | 14:43 |
| sdague | and we're not indexing any ceilometer logs today at all, because they are too big | 14:43 |
| sdague | and a big part of it is this tight polling loop | 14:44 |
| eglynn | sdague: ... is debug verbosity enabled on the ceilo logging in the gate? | 14:44 |
| mtreinish | jogo: heh, your g-r patch doesn't match what you did in the nova hacking bump: https://github.com/openstack/requirements/blob/master/global-requirements.txt#L141 | 14:44 |
| openstackrecheck | Console logs not available after 13:48s for check-tempest-dsvm-postgres-full 92245,5,b3a5ebe | 14:44 |
| sdague | eglynn: debug is enabled for everything | 14:44 |
| *** jaypipes has quit IRC | 14:44 | |
| DinaBelova | sdague, hehe, debug here makes ceilometer to have really lots of logs | 14:45 |
| sdague | DinaBelova: even without debug there are lots of logs | 14:45 |
| DinaBelova | sdague, that's also true | 14:45 |
| sdague | http://logs.openstack.org/05/98505/4/check/check-tempest-dsvm-full/0681279/logs/screen-ceilometer-anotification.txt.gz?level=INFO | 14:45 |
| *** jaypipes has joined #openstack-qa | 14:45 | |
| *** ChanServ sets mode: +o jaypipes | 14:45 | |
| sdague | I would argue that basically none of that is useful to be logged at all | 14:46 |
| eglynn | sdague: so reducing the polling frequency is a short term approach, right? | 14:46 |
| eglynn | sdague: ... more long term we need to be more judicious what we actually log | 14:46 |
| sdague | eglynn: the problem is, you hit the API server just like anything else | 14:47 |
| sdague | we actually want all those transactions from real users | 14:47 |
| sdague | these are actually 2 orthoginal issues really | 14:47 |
| sdague | both of which are currently impacting us | 14:47 |
| eglynn | sdague: hitting the nova-api server, you mean? | 14:48 |
| sdague | or swift-proxy | 14:48 |
| sdague | which is actually where we had to build the work around | 14:48 |
| eglynn | sdague: ... at summit we discussed ways of reducing that polling load on nova-api, but continuing to poll the local hypervisor as before | 14:49 |
| eglynn | sdague: ... for the purposes of the gate, if swift polling is the main issue, would it make to exclude swift metering for now? | 14:50 |
| eglynn | sdague: ... that can be done via the pipeline.yaml | 14:50 |
| sdague | eglynn: ok, there 2 issues | 14:50 |
| sdague | one is the API polling, which we are reducing to ceilometer defaults | 14:50 |
| eglynn | yep | 14:50 |
| sdague | and I would like to actually see if we can get away from eventually | 14:50 |
| sdague | lets say that is "soft fixed" for now | 14:50 |
| eglynn | k | 14:51 |
| sdague | there is a different issue, in that ceilometer itself is generating logging artifacts that are of very dubious value | 14:51 |
| sdague | at a rate which means all the rest of openstack generates about the same amount of artifacts as ceilometer itself | 14:51 |
| sdague | like the fact that ceilometer.pipeline seems to be AUDIT logged every 5 - 10 ms | 14:52 |
| eglynn | yes, I agree, continually reporting that a single sample is published without much context is not valuable | 14:52 |
| sdague | which will also impact the whole system | 14:52 |
| eglynn | sdague: is that 5-10ms frequency of log emission with a polling cadence of 600s? | 14:53 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable E129,E128 rules https://review.openstack.org/98909 | 14:53 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Bump hacking to 0.9.x series https://review.openstack.org/98868 | 14:53 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable E113,E111,H302,F812,E713 rules https://review.openstack.org/98899 | 14:53 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H407,H305,E123,H307,E122 rules https://review.openstack.org/98918 | 14:53 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H402,E251,E265 rules https://review.openstack.org/98903 | 14:53 |
| sdague | eglynn: as far as I can tell it's basically continuous | 14:53 |
| ildikov | eglynn: sdague: BTW was that Logging guideline patch accepted finally in Nova? | 14:53 |
| sdague | ildikov: nope, because I haven't been able to get back to it :) | 14:53 |
| sdague | because, fires. | 14:53 |
| eglynn | ildikov: https://review.openstack.org/#/c/91446/ | 14:54 |
| ildikov | eglynn: sdague: I'm just wondering if that could be a good guide maybe to clean up a bit the logging in Ceilo | 14:54 |
| sdague | that being said, this is a clear issue that was brought up on the list a couple months ago | 14:54 |
| ildikov | sdague: I know that feeling, not just the sun burns these days :) | 14:54 |
| sdague | :) | 14:54 |
| ildikov | sdague: do you mean the swift related logging topic? | 14:54 |
| sdague | ildikov: no, the fact that ceilometer is doing terrible things in logging | 14:55 |
| eglynn | sdague: OK, I'll need to reproduce the issue of 5-10ms emission of log messages with 600s polling, as that does seem indicative of something very wrong | 14:55 |
| sdague | eglynn: well we've got *tons* of data in the gate | 14:55 |
| eglynn | sdague: ... i.e. more than simple over-zealous logging | 14:55 |
| sdague | I'd say just crack open the ceilometer logs for any run | 14:56 |
| *** afazekas has quit IRC | 14:56 | |
| sdague | and decide if that's what you'd want people to be seeing in production | 14:56 |
| sdague | knowing that all production clouds are running at debug at this point | 14:56 |
| sdague | because not doing so means you can't trouble shoot anything :) | 14:57 |
| eglynn | sdague: ... was just gonna ask, would production normally have debug enabled | 14:57 |
| *** mmaglana has joined #openstack-qa | 14:57 | |
| sdague | from summit - yes | 14:57 |
| eglynn | sdague: ... k, got it | 14:57 |
| ildikov | sdague: would that logging guideline be useful for us in its current state, even if it is not accepted yet? | 14:57 |
| sdague | ildikov: yeh, I think there is a ton of stuff in it's that not controversial | 14:57 |
| eglynn | sdague: right-o ... so we've two things to look at with urgency on the ceilo side | 14:57 |
| *** roeyc has quit IRC | 14:58 | |
| sdague | eglynn: awesome :) | 14:58 |
| sdague | thanks | 14:58 |
| ildikov | eglynn: I think on the last logging and monitoring ops session there were a few guys who said yes for debug, even if it is not that good in general... :( | 14:58 |
| eglynn | sdague: 1. rate of log message emission | 14:58 |
| eglynn | sdague: 2. optimization of polling public APIs (e.g. nova-api) | 14:58 |
| sdague | eglynn: yep, swift optimization is probably more important, because swift doesn't use oslo logging, which means filtering is more difficult | 14:59 |
| ildikov | sdague: cool, we will use that guide then, tnx :) | 14:59 |
| sdague | ildikov: thanks | 14:59 |
| eglynn | sdague: cool, got it | 14:59 |
| *** aimon has quit IRC | 15:00 | |
| sdague | sorry for you guys to get the brunt of it here. It's been a thing that came up a few times before, and never got ceilo bites on it. Very happy to see people diving in now. | 15:00 |
| eglynn | sdague: np, we need to get our side of the equation sorted out | 15:01 |
| ildikov | sdague: we had some discussions for instance about logging earlier, we just had even more urgent issues to deal with before that one :( | 15:01 |
| *** mmaglana has quit IRC | 15:01 | |
| sdague | ildikov: totally understand | 15:02 |
| openstackrecheck | Console logs not available after 14:00s for gate-heat-python26 98687,3,2f535a4 | 15:03 |
| ildikov | sdague: we had to fix some things to be able to generate a nice log flood more efficiently ;) | 15:03 |
| sdague | ildikov: nice :) | 15:03 |
| dims_ | sdague, followed the logs for the FloatingIP add request and it's calling iptables-save/restore just fine and commands do suceed, at this point the only thing i can think of is if we can dump the iptables when we hit ssh timeout to see what gives | 15:04 |
| ildikov | sdague: thanks for the heads up on the two main issues here | 15:04 |
| mtreinish | dims_: if you can find an ssh timeout on test_minimum_basic that should already include a iptables dump. | 15:07 |
| mtreinish | I had a log from it earlier let me see if I can dig it up | 15:08 |
| dims_ | mtreinish, ah. ok. | 15:08 |
| *** marun has joined #openstack-qa | 15:08 | |
| *** ildikov_ has joined #openstack-qa | 15:08 | |
| *** ildikov has quit IRC | 15:09 | |
| mtreinish | dims_: http://logs.openstack.org/07/98507/1/check/check-grenade-dsvm-icehouse/ab8c90a/console.html.gz#_2014-06-06_21_06_36_820 | 15:09 |
| dims_ | thanks! will download the whole set | 15:09 |
| *** mmaglana has joined #openstack-qa | 15:12 | |
| *** mkollaro has quit IRC | 15:12 | |
| *** mkollaro has joined #openstack-qa | 15:13 | |
| *** yrabl has quit IRC | 15:15 | |
| openstackgerrit | A change was merged to openstack-dev/grenade: Support Ceilometer services https://review.openstack.org/94468 | 15:15 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Update again and retry a failed package install https://review.openstack.org/98398 | 15:15 |
| openstackgerrit | A change was merged to openstack-dev/devstack: make get-pip.py download failures more clear https://review.openstack.org/98098 | 15:16 |
| sdague | eglynn: ^^^ - the grenade patch merged | 15:17 |
| sdague | mtreinish: where is the patch to move the logging to the end of the file? | 15:17 |
| eglynn | sdague: thank you sir! ... we can declare victory on that BP for juno-1 | 15:18 |
| mtreinish | sdague: this one?: https://review.openstack.org/#/c/97894/ | 15:18 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Change how tempest debug logs are displayed https://review.openstack.org/97894 | 15:19 |
| sdague | mtreinish: yep | 15:21 |
| sdague | mkoderer: you +2 on that ^^^ | 15:21 |
| sdague | ? | 15:21 |
| mtreinish | sdague: the only thing I wasn't sure of on that was I dumped the failure logs before the summary. I wasn't sure which way was better... | 15:23 |
| sdague | mtreinish: yeh, I think the current way seems fine | 15:23 |
| sdague | the icehouse run failed, so you can see it | 15:23 |
| sdague | I was unsure about that either, but I think this probably makes hunting for the fails less burdensome | 15:23 |
| *** atiwari has joined #openstack-qa | 15:24 | |
| sdague | mtreinish: do you know how to inject other attachments into the subunit stream? | 15:24 |
| sdague | I was thinking that is probably what should be done with the failure dumps | 15:25 |
| mtreinish | sdague: not off the top of my head | 15:25 |
| openstackrecheck | Console logs not available after 13:48s for check-swift-dsvm-functional 97339,4,3e79c16 | 15:25 |
| sdague | instead of just inlining it in the logs, creating some new attachments | 15:25 |
| mtreinish | sdague: oh for the console logs, yeah that might be useful | 15:25 |
| sdague | yeh | 15:25 |
| sdague | and for anything like iptables dumps | 15:26 |
| sdague | then it wouldn't fill up the tempest log | 15:26 |
| sdague | but we'd be able to return it on failures | 15:26 |
| sdague | that honestly might be the way to deal with the debug http tracing as well | 15:26 |
| sdague | we'll build kind of big subunit databases, but I'm not sure that matters | 15:27 |
| *** jgrimm has joined #openstack-qa | 15:28 | |
| jordanP | mtreinish, hi. Sorry to bother you, I know now is not the best time. Still working on my shelve/unshelve scenario. So I got my nova patch merged in Havana. But I am facing 2 more small problems that require a backport to havana (again !). The relevant trace in havana is here http://logs.openstack.org/65/96165/1/check/check-grenade-dsvm-partial-ncpu-icehouse/2ac603c/logs/new/screen-n-api.txt.gz#_2014-06-04_12_57_04_476 and the patch I need to get merg | 15:29 |
| jordanP | ed is https://review.openstack.org/#/c/72521/ | 15:29 |
| jordanP | (i know you said you would look, sorry to insist) | 15:30 |
| mtreinish | sdague: http://bazaar.launchpad.net/~python-fixtures/python-fixtures/trunk/view/head:/lib/fixtures/_fixtures/streams.py and http://bazaar.launchpad.net/~python-fixtures/python-fixtures/trunk/view/head:/lib/fixtures/_fixtures/logger.py#L94 | 15:32 |
| mtreinish | that looks like how the log fixture is doing it | 15:32 |
| mtreinish | so building another fixture that adds a new stream is probably what I'd try | 15:32 |
| *** aimon has joined #openstack-qa | 15:33 | |
| *** mkollaro has quit IRC | 15:35 | |
| *** Longgeek has quit IRC | 15:39 | |
| openstackrecheck | Console logs not available after 13:32s for gate-nova-python27 98573,3,b9aec1c | 15:39 |
| mtreinish | jordanP: it looks fine to me | 15:46 |
| mriedem | sdague: mtreinish: looks like dumping server console output on timeout has issues | 15:46 |
| mriedem | http://logs.openstack.org/30/98530/1/check/check-grenade-dsvm/6c88d91/console.html#_2014-06-10_03_52_31_555 | 15:46 |
| mriedem | Details: {u'message': u"Invalid input for field/attribute length. Value: None. None is not of type 'integer', 'string'", u'code': 400} | 15:46 |
| mriedem | i'll check it out | 15:47 |
| jordanP | mtreinish, thanks it helps a lot :) As you might have read in the comments, this patch should not be merged without this one https://review.openstack.org/#/c/77734/ (another backport, only one LOC) that fixes a bug that was introduced by the patch you just +2ed) | 15:49 |
| mtreinish | jordanP: I'm wondering if we should just disable the shelve extension for that job instead of doing a bunch of havana backports | 15:49 |
| jordanP | the person who submited the backports decided not to merge the 2 backports in one | 15:50 |
| jordanP | I haven't concidered this option | 15:50 |
| jordanP | shelving is not a configuration option in tempest, as far as I know | 15:50 |
| jordanP | but there might be a way, maybe | 15:51 |
| mtreinish | jordanP: it's an extension so you can use the api_extensions option | 15:51 |
| jordanP | i'll look into it, anyway thanks a lot ! | 15:51 |
| mtreinish | you just decorate the test saying it requires the shelve extension (which you should probably do anyway) | 15:51 |
| jordanP | yep, good idea, will do | 15:52 |
| jordanP | but then on the gate, this extension should be deactivated "server-side" right ? | 15:52 |
| jordanP | for the job havana-n-partial only | 15:53 |
| jordanP | *n-compute | 15:53 |
| mtreinish | mriedem: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/common/waiters.py#n27 it's the none | 15:53 |
| mtreinish | it needs to be a number | 15:53 |
| mriedem | mtreinish: yeah, fixing it now | 15:53 |
| mriedem | mtreinish: although the nova api docs say length is optional | 15:54 |
| mtreinish | mriedem: so it's probably better to fix the tempest client to allow that | 15:54 |
| mtreinish | jordanP: yeah you'll need a devstack-gate change making the list of available extensions set for that job not include shelve | 15:54 |
| *** catohornet1 has joined #openstack-qa | 15:54 | |
| mtreinish | it's probably set to all right now | 15:54 |
| jordanP | okay | 15:55 |
| openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: Adds "user-password-update" to v3/test_users https://review.openstack.org/92100 | 15:56 |
| *** dustins1 has joined #openstack-qa | 15:57 | |
| *** qba73 has quit IRC | 15:57 | |
| mriedem | mtreinish: so the client blows up because of the jsonschema input validation right? | 15:58 |
| mtreinish | mriedem: I thought that was only on the response side | 15:59 |
| mtreinish | in tempest at least | 15:59 |
| mriedem | yeah that's what it looks like | 15:59 |
| mriedem | so maybe api validation on the nova side is busted | 15:59 |
| mriedem | must be | 15:59 |
| mriedem | ah yup | 15:59 |
| mriedem | it's v3 | 16:00 |
| mriedem | File "tempest/api/compute/v3/servers/test_server_rescue.py", line 30, in setUpClass | 16:00 |
| *** dustins has quit IRC | 16:00 | |
| mtreinish | mriedem: another reason to stop gating on v3 :) | 16:00 |
| mtreinish | I'll send that proposal out today | 16:00 |
| *** coolsvap|afk is now known as coolsvap | 16:01 | |
| *** mlavalle has joined #openstack-qa | 16:01 | |
| dims_ | mtreinish, there's a boatload of "Adding interface eth0 to bridge br100" in the logs you pointed me to | 16:04 |
| dkranz | andreaf: Were you going to update https://review.openstack.org/#/c/86967/ (non-admin)? It would be great to get it approved soon. | 16:05 |
| mtreinish | dims_: so it's probably not the best example. :) It's a grenade run icehouse run so it's going from havana->icehouse... | 16:05 |
| dkranz | andreaf: Also, did my comment a minute ago address your question about the client-checking spec? | 16:05 |
| andreaf | dkranz: I uploaded patchset #7 on non-admin bp | 16:06 |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 92512,6,ed03a4d | 16:06 |
| dims_ | mtreinish, i see the same problem in ther other set of logs i was looking at | 16:06 |
| dkranz | sdague, mkoderer : I responded to your -1 on https://review.openstack.org/#/c/93037/. Can you please remove the -1 or set me straight? | 16:07 |
| mriedem | mtreinish: https://review.openstack.org/#/c/81947/ | 16:07 |
| *** jgrimm has quit IRC | 16:07 | |
| mriedem | have to pass -1 for unlimited in v3 api i guess | 16:07 |
| andreaf | dkranz: or do you mean answer Rochelle's comment? I will do that | 16:07 |
| dkranz | andreaf: That's what I meant. | 16:07 |
| dims_ | mtreinish, linux_net.ensure_bridge does not seem to find br100 properly and tries to rebuild it | 16:08 |
| mtreinish | mriedem: oh, yeah that should work | 16:08 |
| mriedem | mtreinish: easy fix, i'll just update the tempest v3 servers client | 16:08 |
| mtreinish | dims_: hmm, yeah that could be a part of it. Although the check is just looking for the bridge file in /sys/class/net | 16:10 |
| dims_ | mtreinish, will try to understand the code and ping back :) | 16:10 |
| *** evgenyf has quit IRC | 16:13 | |
| *** mmaglana has quit IRC | 16:15 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack/tempest: Convert get_console_output length=None to -1 for Nova V3 API calls https://review.openstack.org/99144 | 16:16 |
| mriedem | mtreinish: sdague: ^ | 16:16 |
| andreaf | dkranz: + 1 on the client-checking | 16:18 |
| dkranz | andreaf: thanks | 16:18 |
| *** jaypipes has quit IRC | 16:20 | |
| andreaf | dkranz: for the non-admin I will do asap, but first I'm trying to find a solution for https://review.openstack.org/#/c/98400/ (static test accounts). | 16:20 |
| openstackrecheck | Console logs not available after 13:28s for check-tempest-dsvm-neutron 98564,3,a6b6c89 | 16:20 |
| dkranz | andreaf: Cool! I guess that is not ready for review yet, right? | 16:21 |
| andreaf | no it's just a problem statement this far - but if you have ideas on how to do it please feel free to comment! | 16:22 |
| *** thomasbiege1 has joined #openstack-qa | 16:25 | |
| *** luqas has quit IRC | 16:25 | |
| *** thomasbiege has quit IRC | 16:26 | |
| jogo | mtreinish: AFAIK it doesn't need to be does it? | 16:27 |
| jogo | mtreinish: if g-r matched per project hacking rules, all projects with hacking 0.8 would fail (since it wouldn't explicitly be in the mirror). | 16:28 |
| *** jgrimm has joined #openstack-qa | 16:29 | |
| *** thomasbiege has joined #openstack-qa | 16:30 | |
| *** thomasbiege1 has quit IRC | 16:30 | |
| *** thomasbiege has quit IRC | 16:31 | |
| *** jaypipes has joined #openstack-qa | 16:33 | |
| *** ChanServ sets mode: +o jaypipes | 16:33 | |
| *** mriedem has quit IRC | 16:35 | |
| *** markmcclain has joined #openstack-qa | 16:36 | |
| openstackrecheck | Console logs not available after 13:56s for gate-tempest-dsvm-large-ops 98947,1,92deed6 | 16:38 |
| *** mmaglana has joined #openstack-qa | 16:41 | |
| mkoderer | dkranz: yep I already tried to discuss with sdague but he was busy with other topics | 16:44 |
| mkoderer | dkranz: I think we should add the detail in the spec | 16:44 |
| mkoderer | but anyway at the end it's just a nit | 16:46 |
| *** mwagner_lap has quit IRC | 16:48 | |
| *** dkranz has quit IRC | 16:49 | |
| *** mriedem has joined #openstack-qa | 16:49 | |
| mtreinish | jogo: but won't the proposal bot keep throwing up commits because they don't match? | 16:49 |
| *** gszasz has quit IRC | 16:50 | |
| jogo | mtreinish: we have a special exception for hacking | 16:50 |
| *** gszasz has joined #openstack-qa | 16:50 | |
| *** giulivo has quit IRC | 16:51 | |
| mtreinish | jogo: really? tempest got one with the bump: https://review.openstack.org/#/c/96829/ | 16:52 |
| jogo | ruh roh | 16:52 |
| jogo | mordred: ^ | 16:52 |
| jogo | fungi: ^ | 16:53 |
| mtreinish | I also like that it fails the requirements check... | 16:53 |
| *** gszasz has quit IRC | 16:54 | |
| jogo | mtreinish: link? | 16:55 |
| *** gszasz has joined #openstack-qa | 16:55 | |
| mtreinish | http://logs.openstack.org/29/96829/7/check/gate-tempest-requirements/d31fec2/console.html | 16:56 |
| eglynn | sdague: FYI https://review.openstack.org/99147 (ceilo: Remove low-value logging from publication codepath) | 16:56 |
| sdague | eglynn: awesome | 16:57 |
| *** leseb has quit IRC | 16:57 | |
| jogo | mtreinish: lol, did your bump hacking patches in tempest land at least? | 16:58 |
| *** gszasz has quit IRC | 16:58 | |
| *** gszasz has joined #openstack-qa | 16:59 | |
| mtreinish | jogo: nope they're stuck on the same thing: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:hacking,n,z | 16:59 |
| mtreinish | in the most recent rev I tried copying the string from g-r and had the same issue | 16:59 |
| mtreinish | does your nova patch get past the requirements check? | 16:59 |
| *** gszasz has quit IRC | 17:00 | |
| jogo | mtreinish: I doubt it | 17:01 |
| *** gszasz has joined #openstack-qa | 17:03 | |
| *** weshay has quit IRC | 17:04 | |
| *** weshay has joined #openstack-qa | 17:04 | |
| *** jpich has quit IRC | 17:04 | |
| *** thomasbiege has joined #openstack-qa | 17:05 | |
| *** eglynn has quit IRC | 17:06 | |
| openstackrecheck | Console logs not available after 13:34s for check-tempest-dsvm-full 85687,6,fd4e06f | 17:07 |
| mordred | jogo: what did I do? | 17:07 |
| *** harlowja_away is now known as harlowja | 17:08 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/92436 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1315095 | 17:08 |
| vrovachev | mtreinish: sdague: Hi, in my patch (https://review.openstack.org/#/c/64136/) XenServer CI permanently falls. Fails 11 tests for images. Is it a known issue? Recheck XenServer CI while check won't pass? | 17:11 |
| fungi | jogo: i know of no special exception in the proposal bot for hacking version updates | 17:17 |
| *** vrovachev has quit IRC | 17:17 | |
| *** jgrimm has quit IRC | 17:19 | |
| *** jordanP has quit IRC | 17:19 | |
| openstackrecheck | Console logs not available after 14:23s for gate-devstack-bash8 98951,1,e03d7b2 | 17:23 |
| *** gszasz has quit IRC | 17:25 | |
| *** markmcclain has quit IRC | 17:25 | |
| *** gszasz has joined #openstack-qa | 17:26 | |
| *** dims_ has quit IRC | 17:28 | |
| *** markmcclain has joined #openstack-qa | 17:29 | |
| *** amotoki_ has quit IRC | 17:37 | |
| *** gszasz has quit IRC | 17:37 | |
| *** gszasz has joined #openstack-qa | 17:38 | |
| *** rdekel has joined #openstack-qa | 17:40 | |
| *** dims_ has joined #openstack-qa | 17:42 | |
| *** bkopilov has joined #openstack-qa | 17:49 | |
| openstackgerrit | A change was merged to openstack/tempest: Use correct base test class for test_stress unit tests https://review.openstack.org/99046 | 17:51 |
| *** ildikov_ has quit IRC | 17:53 | |
| *** bknudson has joined #openstack-qa | 17:53 | |
| *** ildikov_ has joined #openstack-qa | 17:54 | |
| *** malini1 has joined #openstack-qa | 17:54 | |
| openstackrecheck | Console logs not available after 13:44s for gate-nova-python27 98724,5,45d3772 | 17:55 |
| bknudson | is this the place to ask a question about adding a test to tempest? | 17:55 |
| malini1 | hello..I am running into compute test failures in one of my tempest patches. Any pointers to how I can get around this http://logs.openstack.org/44/94944/12/check/check-tempest-dsvm-postgres-full/aad1161/logs/testr_results.html.gz ? | 17:56 |
| malini1 | bknudson: it is :) | 17:56 |
| bknudson | ok, so I'm looking to add a test for identity v3, essentially the GET /v3 response | 17:56 |
| bknudson | looks like there isn't one already | 17:56 |
| openstackrecheck | openstack/python-troveclient change: https://review.openstack.org/90462 failed gate-tempest-dsvm-neutron in the gate queue with an unrecognized error | 17:57 |
| bknudson | but I'm looking at implementing a blueprint and want to make sure the behavior is covered already | 17:57 |
| bknudson | I think I need to update services/identity/v3/json/identity_client.py to add a get_version | 17:57 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint fur bug 1268725 https://review.openstack.org/98913 | 17:57 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Remove duplicate query for 1326820 and 1326855 https://review.openstack.org/98919 | 17:57 |
| bknudson | and add a tempest/api/identity/v3/test_versions.py file with the test in it | 17:57 |
| bknudson | anything else I'd be missing? | 17:57 |
| bknudson | are unit tests required for services/identity/v3/json/identity_client.py ? | 17:58 |
| bknudson | There aren't any existing unit tests for services/identity/v3/json/identity_client.py | 17:58 |
| malini1 | bknudson: AFAIK you don't need unit tests for the client modules. But might be a good idea to confirm with one of the core folks | 17:59 |
| *** jlibosva has quit IRC | 18:01 | |
| *** rwsu has joined #openstack-qa | 18:02 | |
| *** jaypipes has quit IRC | 18:03 | |
| *** gszasz has quit IRC | 18:04 | |
| *** mrodden has joined #openstack-qa | 18:05 | |
| *** ildikov_ has quit IRC | 18:06 | |
| *** openstackrecheck has quit IRC | 18:08 | |
| *** openstackrecheck has joined #openstack-qa | 18:08 | |
| *** leseb has joined #openstack-qa | 18:08 | |
| *** leseb has quit IRC | 18:13 | |
| *** coolsvap is now known as coolsvap|afk | 18:17 | |
| *** __afazekas is now known as afazekas | 18:18 | |
| afazekas | sdague: we timeout the ssh connection after 8 sec http://logs.openstack.org/31/95831/4/check/check-grenade-dsvm/421ce32/console.html#_2014-06-09_15_35_58_878 , we should retry it https://review.openstack.org/#/c/98212/ | 18:19 |
| sdague | afazekas: no we don't | 18:20 |
| sdague | you aren't reading the minutes column | 18:20 |
| afazekas | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcInRlbXBlc3QuY29tbW9uLnNzaCB0aW1lb3V0XFw6IHRpbWVkIG91dFwiIEFORCB0YWdzOlwiY29uc29sZVwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDAyNDI0NDA3NDA5LCJtb2RlIjoiIiwiYW5hbHl6ZV9maWVsZCI6IiJ9 | 18:20 |
| sdague | we time out after 3 minutes and 8 seconds | 18:20 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Refine queries for tripleo jobs https://review.openstack.org/98096 | 18:20 |
| sdague | that tripped me up the other day as well | 18:21 |
| sdague | but the timestamps are very clear | 18:21 |
| afazekas | sdague: I do not really like the realtive time print :) | 18:21 |
| afazekas | thx | 18:21 |
| sdague | now, why we are sleeping an extra 60 seconds, that's probably a bug | 18:22 |
| afazekas | sdague: the 60 sec sleep was in the tcp socket | 18:22 |
| afazekas | The retry is good as it is :( | 18:23 |
| afazekas | just the 'after 2 seconds' printing is incorrect | 18:23 |
| sdague | yeh | 18:23 |
| sdague | if you want to fix the message, I'd be +2 on that | 18:23 |
| sdague | better to not confuse people | 18:24 |
| *** harlowja is now known as harlowja_away | 18:25 | |
| afazekas | How could we make the gating issues more debug-able ? | 18:25 |
| *** dims_ has quit IRC | 18:25 | |
| *** openstackrecheck has quit IRC | 18:26 | |
| *** openstackrecheck has joined #openstack-qa | 18:26 | |
| afazekas | I have seen many issues which without local reproduce and trace cannot be solved | 18:29 |
| *** safchain has quit IRC | 18:29 | |
| afazekas | sdague: What else could be logged,traced, dumped , what trick could be used on failure ? | 18:30 |
| *** ildikov has joined #openstack-qa | 18:33 | |
| *** jgrimm has joined #openstack-qa | 18:33 | |
| *** kgriffs is now known as kgriffs|afk | 18:35 | |
| *** nati_ueno has joined #openstack-qa | 18:35 | |
| openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm 92166,15,67211c9 | 18:39 |
| sdague | afazekas: so this was the conversation we started having yesterday, though I'm about to be less useful for the rest of the day. What I think we really need is a unified verification class for guests | 18:40 |
| sdague | that includes a series of debug operations | 18:40 |
| afazekas | sdague: what else than console output check ? | 18:41 |
| sdague | afazekas: icmp traffic works | 18:42 |
| sdague | do another round of collecting floating ip config | 18:42 |
| sdague | I'm not entirely sure yet | 18:43 |
| *** wramthun has quit IRC | 18:43 | |
| sdague | however if we had it all consolidated we could make a pretty thurough dump | 18:43 |
| afazekas | On all vm, or just when the ssh connection fails ? | 18:43 |
| *** wramthun has joined #openstack-qa | 18:43 | |
| afazekas | https://github.com/openstack/tempest/blob/master/tempest/common/ssh.py#L69 her we could add an extra try which finds the vm related to the ip and does the stuff | 18:44 |
| afazekas | All vm would have time cost, Very frequently we delete the vm even without an active state (~ 2 sec lifetime), deleting a vm in active state ~12 sec lifetime. deleting a pingable vm can be more than 30 sec | 18:49 |
| *** SridharG has quit IRC | 18:51 | |
| openstackrecheck | Console logs not available after 13:31s for check-tempest-dsvm-postgres-full 95280,3,69d89b4 | 18:53 |
| *** dustins1 has left #openstack-qa | 18:54 | |
| *** dustins1 has joined #openstack-qa | 18:54 | |
| *** dustins1 has left #openstack-qa | 18:56 | |
| *** dustins1 has joined #openstack-qa | 18:56 | |
| *** markmcclain has quit IRC | 18:56 | |
| *** nati_ueno has quit IRC | 18:57 | |
| *** dustins1 has left #openstack-qa | 18:57 | |
| *** dustins1 has joined #openstack-qa | 18:57 | |
| *** marun is now known as marun_afk | 18:59 | |
| *** SlickN1k has quit IRC | 19:01 | |
| *** bknudson has quit IRC | 19:02 | |
| *** aimon has quit IRC | 19:02 | |
| *** harlowja_away is now known as harlowja | 19:03 | |
| openstackrecheck | Console logs not available after 13:29s for gate-nova-python26 99153,1,2c4d1bb | 19:06 |
| *** nati_ueno has joined #openstack-qa | 19:09 | |
| *** dims_ has joined #openstack-qa | 19:10 | |
| *** qba73 has joined #openstack-qa | 19:11 | |
| *** qba73 has quit IRC | 19:15 | |
| *** dustins1 has quit IRC | 19:16 | |
| openstackrecheck | Console logs not available after 13:29s for gate-nova-pep8 99157,1,dc5faa8 | 19:20 |
| *** amotoki has quit IRC | 19:22 | |
| *** afazekas has quit IRC | 19:26 | |
| *** Sukhdev has joined #openstack-qa | 19:31 | |
| *** catohornet1 has quit IRC | 19:31 | |
| openstackrecheck | Console logs not available after 13:24s for gate-nova-pep8 99155,1,5571911 | 19:33 |
| *** malini2 has joined #openstack-qa | 19:34 | |
| *** malini1 has quit IRC | 19:37 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-dev/devstack: Expand variables in get_meta_section_files https://review.openstack.org/98951 | 19:39 |
| *** Sukhdev has quit IRC | 19:40 | |
| openstackrecheck | Console logs not available after 13:30s for gate-nova-pep8 99154,1,da5d27e | 19:47 |
| *** bknudson has joined #openstack-qa | 19:50 | |
| *** marun_afk is now known as marun | 19:55 | |
| *** jaypipes has joined #openstack-qa | 20:00 | |
| *** ChanServ sets mode: +o jaypipes | 20:00 | |
| openstackrecheck | Console logs not available after 13:51s for gate-nova-pep8 99156,1,2a0bebc | 20:01 |
| *** markmcclain has joined #openstack-qa | 20:02 | |
| *** aimon_ has joined #openstack-qa | 20:10 | |
| *** nati_ueno has quit IRC | 20:11 | |
| openstackrecheck | Console logs not available after 13:24s for gate-heat-python26 88636,6,bd150fb | 20:14 |
| *** cbader has joined #openstack-qa | 20:17 | |
| *** nati_ueno has joined #openstack-qa | 20:19 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/qa-specs: Spec for running tempest as non-admin https://review.openstack.org/86967 | 20:20 |
| andreaf | dkranz: ^^^ | 20:21 |
| mriedem | sdague: mtreinish: i'm smelling something in the 3rd party boto tests possibly related to nova-network leaks | 20:22 |
| mriedem | http://goo.gl/6f1dfw | 20:23 |
| mriedem | seeing InstanceNotFound in allocate_fixed_ip for the boto instance run tests | 20:23 |
| mriedem | http://git.openstack.org/cgit/openstack/tempest/tree/tempest/thirdparty/boto/test_ec2_instance_run.py | 20:23 |
| mriedem | dims_: ^ | 20:23 |
| mriedem | wondering if something in the ec2 api flow isn't cleaning up networks? | 20:23 |
| mriedem | mikal: ^ since i found that looking at your logging patch | 20:24 |
| *** afazekas has joined #openstack-qa | 20:24 | |
| sdague | mriedem: could be, it happens really late though | 20:25 |
| sdague | the ec2 tests that is | 20:25 |
| mriedem | around the same time as scenario tests though right? | 20:26 |
| mriedem | and that's when we also see the ssh failures | 20:26 |
| openstackrecheck | Console logs not available after 13:26s for gate-nova-pep8 94915,11,de9a324 | 20:28 |
| *** kmartin has quit IRC | 20:33 | |
| *** enikanorov_ has quit IRC | 20:33 | |
| *** esmute has quit IRC | 20:33 | |
| *** cschwede_ has quit IRC | 20:33 | |
| *** Sukhdev has joined #openstack-qa | 20:33 | |
| *** esmute has joined #openstack-qa | 20:33 | |
| *** kmartin has joined #openstack-qa | 20:33 | |
| *** kmartin has quit IRC | 20:33 | |
| *** kmartin has joined #openstack-qa | 20:33 | |
| *** cschwede has joined #openstack-qa | 20:35 | |
| *** plomakin_ has joined #openstack-qa | 20:39 | |
| openstackrecheck | Console logs not available after 13:34s for gate-nova-pep8 94501,7,3d39e05 | 20:41 |
| *** dhellmann_ has joined #openstack-qa | 20:41 | |
| *** kgriffs|afk is now known as kgriffs | 20:46 | |
| *** dhellmann_ has quit IRC | 20:48 | |
| *** dhellmann_ has joined #openstack-qa | 20:48 | |
| *** sballe has joined #openstack-qa | 20:49 | |
| sdague | mriedem: yeh often | 20:52 |
| sdague | mriedem: it's a good guess, the ec2 paths are kind of gorp | 20:52 |
| sdague | in both nova and tempest | 20:53 |
| sdague | mriedem loves his bug numbers :) | 20:53 |
| andreaf | mtreinish: ping | 20:53 |
| mriedem | now if i only knew anything about the ec2 api | 20:54 |
| mtreinish | andreaf: pong | 20:54 |
| mriedem | sdague: i certainly do, otherwise i get lost | 20:54 |
| *** bknudson has quit IRC | 20:54 | |
| *** rdekel has quit IRC | 20:54 | |
| *** mmaglana has quit IRC | 20:54 | |
| *** dhellmann has quit IRC | 20:54 | |
| *** Obulpathi has quit IRC | 20:54 | |
| *** masayukig has quit IRC | 20:54 | |
| *** coolsvap|afk has quit IRC | 20:54 | |
| *** smurashov has quit IRC | 20:54 | |
| *** plomakin has quit IRC | 20:54 | |
| *** openstackgerrit has quit IRC | 20:54 | |
| *** dhellmann_ is now known as dhellmann | 20:54 | |
| andreaf | mtreinish: what's the policy for using the gate tag on tests? https://review.openstack.org/#/c/90306/10/tempest/api/volume/admin/test_qos.py,cm | 20:54 |
| *** smurashov has joined #openstack-qa | 20:54 | |
| sdague | andreaf: the policy should be that we remove it | 20:54 |
| mtreinish | andreaf: I don't think there is one, it doesn't actually do anything | 20:54 |
| sdague | it's vestigial from an effort that never happened | 20:54 |
| andreaf | so no no gate tag on new test | 20:55 |
| andreaf | sdague, mtreinish: thanks | 20:55 |
| openstackrecheck | Console logs not available after 13:30s for gate-nova-pep8 95279,6,0fe288b | 20:55 |
| andreaf | it could worth a note in hacking or ~ | 20:56 |
| *** nati_ueno has quit IRC | 20:56 | |
| *** dhellmann has quit IRC | 20:56 | |
| *** rdekel has joined #openstack-qa | 20:57 | |
| *** enikanorov_ has joined #openstack-qa | 20:59 | |
| *** nati_ueno has joined #openstack-qa | 21:05 | |
| *** mwagner_lap has joined #openstack-qa | 21:06 | |
| openstackrecheck | Console logs not available after 13:32s for check-grenade-dsvm 93754,6,7f76538 | 21:08 |
| *** leseb has joined #openstack-qa | 21:10 | |
| *** rockyg has joined #openstack-qa | 21:10 | |
| *** bknudson has joined #openstack-qa | 21:11 | |
| *** mmaglana has joined #openstack-qa | 21:11 | |
| *** Obulpathi has joined #openstack-qa | 21:11 | |
| *** masayukig has joined #openstack-qa | 21:11 | |
| *** coolsvap|afk has joined #openstack-qa | 21:11 | |
| *** openstackgerrit has joined #openstack-qa | 21:11 | |
| *** nati_ueno has quit IRC | 21:15 | |
| *** mlavalle has quit IRC | 21:21 | |
| openstackrecheck | Console logs not available after 13:25s for gate-ceilometer-python26 77845,36,9641b1b | 21:22 |
| *** leseb has quit IRC | 21:26 | |
| *** jaypipes has quit IRC | 21:26 | |
| *** jaypipes has joined #openstack-qa | 21:29 | |
| *** ChanServ sets mode: +o jaypipes | 21:29 | |
| *** adalbas has quit IRC | 21:29 | |
| *** Obulpathi has quit IRC | 21:32 | |
| *** nati_ueno has joined #openstack-qa | 21:35 | |
| openstackrecheck | Console logs not available after 13:25s for gate-tempest-dsvm-neutron 97071,4,0856433 | 21:35 |
| *** leseb has joined #openstack-qa | 21:36 | |
| openstackgerrit | Carlos Gonçalves proposed a change to openstack-dev/devstack: Support for Neutron Traffic Steering service plugin https://review.openstack.org/99213 | 21:36 |
| *** jaypipes has quit IRC | 21:37 | |
| *** mriedem has quit IRC | 21:47 | |
| *** dims__ has joined #openstack-qa | 21:52 | |
| *** dims_ has quit IRC | 21:54 | |
| *** dhellmann has joined #openstack-qa | 21:55 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-postgres-full 98564,3,9f9e37b | 22:02 |
| *** rdekel has quit IRC | 22:03 | |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: make logformat more similar to oslo https://review.openstack.org/98819 | 22:09 |
| catohornet | n | 22:11 |
| catohornet | n | 22:11 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add a hacking check to block scenario manager in api tests https://review.openstack.org/96164 | 22:13 |
| sballe | hi | 22:13 |
| sballe | I am running into problems with devstack and stable/icehouse. Is this IRC a good place to ask questions? | 22:14 |
| *** nati_ueno has quit IRC | 22:14 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-full 97124,4,d937eb8 | 22:16 |
| *** ekhugen has quit IRC | 22:21 | |
| *** bknudson has quit IRC | 22:22 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 22:24 |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: Use identity_uri instead of auth fragments https://review.openstack.org/94545 | 22:28 |
| *** Sukhdev has quit IRC | 22:29 | |
| *** Sukhdev has joined #openstack-qa | 22:29 | |
| openstackrecheck | Console logs not available after 13:29s for gate-tempest-dsvm-large-ops 85210,2,6337c7c | 22:29 |
| *** leseb has quit IRC | 22:32 | |
| *** nati_ueno has joined #openstack-qa | 22:32 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Bump baremetal power/assoc/unprovision timeouts https://review.openstack.org/99226 | 22:38 |
| openstackrecheck | Console logs not available after 13:42s for gate-trove-python26 94484,4,64f06ac | 22:43 |
| *** rockyg has quit IRC | 22:48 | |
| openstackgerrit | Chris Buccella proposed a change to openstack/tempest: Remove the skip for test_get_instance_action() https://review.openstack.org/99231 | 22:57 |
| openstackrecheck | Console logs not available after 13:51s for check-grenade-dsvm 89474,15,78ce80d | 22:57 |
| dims__ | sdague, mreidem, mtreinish - boto had a release on May 30th (2.29.1) we do seem to be picking it up (http://logs.openstack.org/72/98872/2/check/check-tempest-dsvm-neutron/04fb1fa/logs/pip-freeze.txt.gz) | 22:57 |
| *** rbak has quit IRC | 23:02 | |
| openstackgerrit | A change was merged to openstack/tempest: Test script for CRUD operations on regions https://review.openstack.org/93347 | 23:06 |
| *** dims__ has quit IRC | 23:06 | |
| *** aimon_ has quit IRC | 23:06 | |
| *** cbader has quit IRC | 23:07 | |
| *** dims__ has joined #openstack-qa | 23:08 | |
| *** nati_ueno has quit IRC | 23:10 | |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm-partial-ncpu 95407,3,252edda | 23:10 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: make logformat more similar to oslo https://review.openstack.org/98819 | 23:14 |
| *** atiwari has quit IRC | 23:16 | |
| *** kgriffs is now known as kgriffs|afk | 23:35 | |
| openstackrecheck | Console logs not available after 13:26s for check-grenade-dsvm 98212,1,2b3e6a9 | 23:37 |
| openstackgerrit | Ryan Hsu proposed a change to openstack/tempest: Handle backup not found in test_create_backup https://review.openstack.org/98693 | 23:40 |
| *** yamahata has quit IRC | 23:46 | |
| *** mmaglana has quit IRC | 23:47 | |
| openstackrecheck | Console logs not available after 13:24s for gate-nova-python26 99181,1,16b08fa | 23:50 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer test_notify_alarm bug 1321826 https://review.openstack.org/98149 | 23:55 |
| *** GMann has joined #openstack-qa | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!