Friday, 2014-04-11

*** radsy has quit IRC00:00
*** radsy has joined #openstack-ironic00:01
openstackgerritA change was merged to openstack/ironic-python-agent: Add BackOffLoopingCall with jitter  https://review.openstack.org/8430300:07
*** matsuhashi has joined #openstack-ironic00:24
*** krtaylor has joined #openstack-ironic00:28
openstackgerritRuby Loo proposed a change to openstack/ironic: Install guide for Ironic  https://review.openstack.org/8164500:35
openstackgerritJay Faulkner proposed a change to openstack/ironic-python-agent: Implement version 2 payload with more hw info  https://review.openstack.org/8647500:38
*** killer_prince has joined #openstack-ironic00:39
*** eghobo has joined #openstack-ironic00:42
openstackgerritA change was merged to openstack/ironic-python-agent: Use a released version of Pecan  https://review.openstack.org/8672900:46
*** epim has quit IRC00:58
*** ilives has joined #openstack-ironic01:04
*** ilives has quit IRC01:05
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add agent fields to virt driver  https://review.openstack.org/8513101:10
*** ilives has joined #openstack-ironic01:15
*** epim has joined #openstack-ironic01:23
*** epim has quit IRC01:25
*** newell_ has quit IRC01:31
openstackgerritA change was merged to openstack/ironic: Factor hash ring management out of the conductor  https://review.openstack.org/8481401:37
openstackgerritA change was merged to openstack/ironic: Updated from global requirements  https://review.openstack.org/8582001:45
*** nosnos has joined #openstack-ironic01:51
*** eghobo has quit IRC01:57
openstackgerritA change was merged to openstack/python-ironicclient: Fix docstring for client.get_client()  https://review.openstack.org/8609501:58
*** rloo has quit IRC02:13
*** coolsvap|afk is now known as coolsvap02:18
*** wendar_ is now known as wendar02:21
*** mynameisdeleted has quit IRC02:33
*** mynameisdeleted has joined #openstack-ironic02:34
*** hemna_ has quit IRC02:38
*** jgrimm has joined #openstack-ironic02:44
*** matsuhashi has quit IRC03:14
*** harlowja is now known as harlowja_away03:14
*** nosnos has quit IRC03:20
*** nosnos has joined #openstack-ironic03:20
*** nosnos has quit IRC03:25
*** matsuhashi has joined #openstack-ironic03:28
openstackgerritJenkins proposed a change to openstack/ironic-python-agent: Updated from global requirements  https://review.openstack.org/8523303:42
*** coolsvap is now known as coolsvap|afk03:46
*** rameshg87 has joined #openstack-ironic03:54
*** epim has joined #openstack-ironic03:55
*** coolsvap|afk is now known as coolsvap03:59
*** eghobo has joined #openstack-ironic04:03
*** stillbert has quit IRC04:12
*** epim has quit IRC04:13
*** mikal has joined #openstack-ironic04:14
*** nosnos has joined #openstack-ironic04:14
*** mikal is now known as stillbert04:14
*** pradipta_away has quit IRC04:15
openstackgerritJenkins proposed a change to openstack/python-ironicclient: Updated from global requirements  https://review.openstack.org/8677404:16
rameshg87hello05:01
rameshg87is anyone working on ironic teeth agent in irc right now ? :-)05:01
*** eghobo has quit IRC05:03
*** eghobo has joined #openstack-ironic05:03
Haomengrameshg87: I think the teeth agent bp is this one - https://blueprints.launchpad.net/ironic/+spec/agent-driver05:06
Haomengrameshg87: current owner is Josh Gachnang05:07
*** eguz has joined #openstack-ironic05:09
*** eghobo has quit IRC05:09
*** saju_m has joined #openstack-ironic05:16
rameshg87hello Haomeng:05:20
rameshg87thanks, yeah i got to know that05:20
Haomengrameshg87: :)05:20
Haomengrameshg87: welcome:)05:20
rameshg87i just got the code for the python-ironic-agent and ironic teeth driver05:20
rameshg87just wanted to give a try with what is available now05:20
rameshg87i wanted to check if they have written some etherpad as to how to use it05:20
rameshg87do you have some idea ?05:20
Haomengrameshg87:  teeth driver is portted from other project, maybe we can find the documents, let me try to search05:24
Haomengrameshg87: https://github.com/rackerlabs/ironic-teeth-driver  https://github.com/rackerlabs/teeth-agent05:25
Haomengrameshg87: and this one - https://blueprints.launchpad.net/ironic/+spec/agent-driver05:27
Haomengrameshg87: it is just a deployment tool, it is ramdisk to run on deployment time to get commnuitation between ironic server and barematel node05:28
Haomengrameshg87: not sure if we have etherpad:)05:30
rameshg87yeah i saw this ..05:35
rameshg87Haomeng: but i was just wondering how do i build the ironic-teeth-agent in the ramdisk05:36
*** radsy has quit IRC06:00
*** pradipta has joined #openstack-ironic06:02
*** lsmola has quit IRC06:04
openstackgerritJenkins proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/8395606:07
Haomengrameshg87: it should be built by tripleo I think06:11
Haomengrameshg87: but not sure, we can check with the owner06:12
*** vkozhukalov has joined #openstack-ironic06:29
*** dshulyak has quit IRC06:30
*** vkozhukalov has quit IRC06:31
*** killer_prince has quit IRC06:43
*** manslaughter is now known as mrda_away07:01
*** pbrooko has joined #openstack-ironic07:02
*** lsmola has joined #openstack-ironic07:02
*** mynameisdeleted has quit IRC07:05
*** jistr has joined #openstack-ironic07:14
*** pbrooko has quit IRC07:15
*** killer_prince has joined #openstack-ironic07:16
*** yuriyz has joined #openstack-ironic07:26
*** blamar has quit IRC07:31
*** matsuhashi has quit IRC07:35
*** matsuhashi has joined #openstack-ironic07:35
*** ndipanov has joined #openstack-ironic07:37
*** eguz has quit IRC07:39
*** ifarkas has joined #openstack-ironic07:41
*** romcheg1 has joined #openstack-ironic07:43
*** athomas has quit IRC07:54
Mikhail_D_wkGood morning all! :)07:55
HaomengMikhail_D_wk: morning:)07:56
openstackgerritYuriy Zveryanskyy proposed a change to openstack/ironic: Do not sync power state for new invalidated nodes  https://review.openstack.org/8652908:00
dtantsurMorning Ironic08:05
*** derekh has joined #openstack-ironic08:12
*** vkozhukalov has joined #openstack-ironic08:16
*** pradipta is now known as pradipta_away08:19
*** pradipta_away is now known as pradipta08:19
*** trapsig has joined #openstack-ironic08:24
*** matsuhas_ has joined #openstack-ironic08:25
*** matsuhashi has quit IRC08:25
*** lucasagomes has joined #openstack-ironic08:26
trapsigHello Ironic, so I was trying out Ironic via DevStack and tried booting an instance. Got a FailedToUpdateDHCPOptOnPort Error. How do I get around this error? Appreciate any input. Here are the logs - http://paste.openstack.org/show/75537/08:32
*** lazy_prince has joined #openstack-ironic08:40
Haomengtrapsig: can you check your neutron process08:43
Haomengtrapsig: your log indicates  "ConnectionFailed: Connection to neutron failed: Maximum attempts reached", which means the neutron issue08:44
Haomengtrapsig: try to run " neutron agent-list" to see the neutron agent status08:45
*** mkerrin has quit IRC08:46
Haomengtrapsig: dhcp port is prepared by neutron dhcp agent08:46
Haomengtrapsig: hope that helps you:)08:46
trapsigHaomeng: Yep, q-svc is running08:47
*** dtantsur is now known as dtantsur|lunch08:47
Haomengtrapsig: run command  " neutron agent-list" t to check neutron agents status08:47
*** martyntaylor has joined #openstack-ironic08:47
*** pradipta is now known as pradipta_away08:47
*** max_lobur has joined #openstack-ironic08:47
trapsigHaomeng: All Smileys! :-(08:48
trapsigI mean :-)08:48
Haomengtrapsig: :)08:49
trapsigalive = :-) and admin_state_up = True for neutron agent-list08:49
*** pradipta_away is now known as pradipta08:49
trapsigHaomeng: but still it fails to update neutron08:49
trapsigHaomeng: I actually created a separate bridge and set HOST_IP in localrc to the bridge ip.08:50
trapsigHaomeng: If that helps08:50
Haomengtrapsig: did you run 'neutron agent-list' in ironic node?08:50
HaomengHaomeng: or telnet the neutron api port to check if it is networking issue between neutron api service node and your ironic node08:51
trapsigHaomeng: I'm running via DevStack08:51
Haomengtrapsig: all-in-one env?08:51
trapsigHaomeng: So what do you mean by 'ironic node'? Aren't all the services running in the same node?08:52
*** mkerrin has joined #openstack-ironic08:52
trapsigyep, all in one env!08:52
Haomengtrapsig: ok, see08:52
Haomengtrapsig: can you paste more log, before this line " Command stderr is ..."08:53
Haomengtrapsig: have to check which command is running08:53
Haomengtrapsig: will be back for a while...08:54
trapsigHaomeng: Pasting now!08:54
trapsigOk08:55
Haomengtrapsig: ok08:55
Haomenglet me take a look first:)08:55
trapsighttp://paste.openstack.org/show/75540/08:55
trapsigHaomeng: ^08:55
Haomengok08:55
Haomengtrapsig: looks like there are some other issue - "env LC_ALL=C LANG=C qemu-img info"08:57
Haomengtrapsig: sorry, will be away...08:57
Haomengtrapsig: will be back to help you:)08:57
trapsigHaomeng: That's an error?08:58
trapsigHaomeng: No Problem :-) Sure!08:58
*** lazy_prince has quit IRC09:08
*** max_lobur has quit IRC09:08
*** lazy_prince has joined #openstack-ironic09:09
openstackgerritA change was merged to openstack/ironic: Cleanup running conductor services in tests  https://review.openstack.org/8647309:10
*** rameshg871 has joined #openstack-ironic09:12
*** rameshg87 has quit IRC09:15
trapsigHaomeng: Found the error https://github.com/openstack/ironic/blob/master/ironic/common/neutron.py#L29 . My Neutron endpoint default-ed to 127.0.0.1. Possibly because I changed my HOST_IP in localrc to the bridge IP and the services weren't hosted on 127.0.0.1 but rather on my bridge ip address?09:17
*** rameshg871 has quit IRC09:17
*** martyntaylor has quit IRC09:20
trapsigHaomeng: Now, Ironic is stuck in wait call-back! Checking the console shows the guest instance failed to get a DHCP address. Hmmm! More debugging! Yay!09:20
*** martyntaylor has joined #openstack-ironic09:23
openstackgerritA change was merged to openstack/ironic: Sync object list stuff from Nova  https://review.openstack.org/8669709:27
openstackgerritYuriy Zveryanskyy proposed a change to openstack/ironic: Add worker threads limit to _check_deploy_timeouts task  https://review.openstack.org/7865109:27
openstackgerritA change was merged to openstack/ironic: Make tests use Node object instead of dict  https://review.openstack.org/8486009:27
*** martyntaylor has quit IRC09:30
*** martyntaylor has joined #openstack-ironic09:30
*** martyntaylor has quit IRC09:32
*** overlayer has joined #openstack-ironic09:37
*** jistr has quit IRC09:41
*** martyntaylor has joined #openstack-ironic09:41
Haomengtrapsig: cool:)09:43
*** saju_m has quit IRC09:45
trapsigHaomeng: So now I now have Networking issues :-( ! tcpdump on brbm shows that i get DHCP Discover packets, and I'm guessing no Offers are being made back by Neutron, so the guest instance times out trying to PXE Boot and dies :(09:50
*** max_lobur has joined #openstack-ironic09:51
Haomengtrapsig: can you check dnsmasq process09:54
Haomengto see if neutron launch it correctlly with your mac address in the white list09:55
Haomengtrapsig: and to check if dnsmasq process is binding with correct tap device09:56
Haomengtrapsig: which tap should have correct ip09:56
trapsigHaomeng: its binding to the tap in the dhcp namespace09:57
Haomengtrapsig: check dnsmasq process to see the parameter such as "--bind-interfaces --interface=tapbdbb56de-f8"09:58
*** athomas has joined #openstack-ironic09:58
trapsigHaomeng: Yep, its exactly like that!09:58
Haomengand run "ovs-vsctl show" and "ifconfig -a" to check the tap09:59
Haomengtrapsig: it should have a correct ip which is serving for dhcp request09:59
trapsigthat tap is bound to the br-int bridge09:59
Haomengand response09:59
Haomengtrapsig: that is correct behavior10:00
Haomengtrapsig: check with 'ifconfig -a' to see if the tap has correct ip10:00
Haomengtrapsig: and run tcpdump to capture the dhcp packages with tap port10:00
trapsigHaomeng: So i checked ifconfig -a10:01
Haomengtrapsig: if we can see the dhcp response, that  should work for neutron10:01
trapsigthe tap didn't exist! but i do ip netns exec qdhcp-blah ifconfig -a10:01
trapsigthe tap exists10:01
*** jistr has joined #openstack-ironic10:01
*** saju_m has joined #openstack-ironic10:02
trapsigHaomeng: and ovs-tap1 and brbm-tap1 which exist in the regular namespace don't have any IP addresses!10:02
Haomengtrapsig: did you run devstack with refresh env?10:02
trapsigyep with RECLONE=Yes10:03
Haomengtrapsig: it is better we delete/remove all port/tap from neutron before devstack stack.sh10:03
*** nosnos has quit IRC10:03
trapsigHaomeng: Ah, I will try that! By the way, from neutron or OVS? Neutron anyway gets purged right?10:04
*** matsuhas_ has quit IRC10:04
trapsigHaomeng: Or it doesnt?10:04
Haomengtrapsig: not sure, we can remove them by manually first10:07
Haomengtrapsig: but still not sure the root cause, have to check neutron log10:08
Haomengtrapsig: and have a try, good luck10:08
Haomengtrapsig: :)10:09
trapsigHaomeng: I'll poke more into Neutron then and come back! Thanks a lot! :-)10:10
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add ManagementInterface  https://review.openstack.org/8606310:11
Haomengtrapsig: welcome:)10:12
trapsigHaomeng: Just need to figure out why neutron won't send DHCP Offers! Must be a dnsmasq issue like you said! So I manually fired up a dnsmasq on brbm and the instance got an IP address! Anyway, thanks again! You've helped me a lot! ^^10:13
Haomengtrapsig: the dhcp response is sent from dnsmasq to tap device10:13
Haomengtrapsig: neutron just help to setup dnsmasq process to serve the mac for us10:15
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMITool to use the new ManagementInterface  https://review.openstack.org/8609210:16
*** martyntaylor1 has joined #openstack-ironic10:16
*** martyntaylor has quit IRC10:17
trapsigHaomeng: Sorry if this question may seem stupid, but how is the baremetal instance related/connected to the tap device on the qdhcp network namespace? I can't seem to connect the dots here :-( Anywhere I could read up on it?10:17
agordeevtrapsig: regarding ovs-tap1 and brbm-tap1. It's just veth pair between two bridges. If their links are set up, they should be okay.10:18
agordeevmorning Ironic :)10:18
agordeevtrapsig: let me help you.10:18
trapsigagordeev: Sure! :-)10:19
*** yuriyz has quit IRC10:19
trapsigagordeev: tl; dr My baremetal instance won't get an IP address!10:20
*** athomas has quit IRC10:22
trapsigagordeev: Also, I've manually assigned an IP address to brbm and I'm setting HOST_IP in devstack to that IP address! I keep switching between networks, so I need a single endpoint rather than running stack.sh again every time!10:22
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: SeaMicro to use the new ManagementInterface  https://review.openstack.org/8632810:22
*** yuriyz has joined #openstack-ironic10:24
trapsigagordeev: Not setting HOST_IP didn't help either! Same issue!10:24
agordeevtrapsig: there is no need of assigning any IP to bridges like brbm10:24
*** saju_m has quit IRC10:25
trapsigagordeev: Yup, I understand there isn't! It was only for my convenience as DevStack selects eth0's IP address by default! And my eth0 IP address changes because I keep moving around between different networks :-( Is there a better way to keep a consistent endpoint?10:27
agordeevtrapsig: well, i'll be afk for 30mins. let's continue later.10:28
trapsigagordeev: Sure :-) Thanks!10:28
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMINative to use the new ManagementInterface  https://review.openstack.org/8658810:29
*** k4n0 has joined #openstack-ironic10:41
k4n0Hey  folks, i got a weird problem10:41
k4n0Ironic's deployment ramdisk created using diskimage-builder does not detect any sata disks on the baremetal10:42
*** k4n0 has quit IRC10:42
*** k4n0 has joined #openstack-ironic10:43
k4n0But if pxe-boot a ubuntu image, i can see all the sata devices10:43
*** k4n0 has quit IRC10:45
Haomengk4n0: ramdisk does not contain sata disk driver I think10:59
Haomengk4n0: what is your sata mode? achi or sata ?11:01
openstackgerritA change was merged to openstack/ironic: Supports filtering port by address  https://review.openstack.org/8277311:04
*** mcdonagm has joined #openstack-ironic11:05
*** mcdonagm has quit IRC11:08
openstackgerritVladimir Kozhukalov proposed a change to openstack/ironic-python-agent: Added disk partitioner  https://review.openstack.org/8616311:14
*** pradipta is now known as pradipta_away11:20
*** dtantsur|lunch is now known as dtantsur11:21
openstackgerritVladimir Kozhukalov proposed a change to openstack/ironic-python-agent: Added disk partitioner  https://review.openstack.org/8616311:23
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add DiskPartitioner  https://review.openstack.org/8339611:34
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Use DiskPartitioner  https://review.openstack.org/8339911:34
*** max_lobur1 has joined #openstack-ironic11:42
*** max_lobur has quit IRC11:42
*** lucasagomes is now known as lucas-hungry11:44
*** max_lobur has joined #openstack-ironic11:45
*** max_lobur1 has quit IRC11:45
*** romcheg1 has quit IRC11:46
*** coolsvap is now known as coolsvap|afk11:47
openstackgerritVladimir Kozhukalov proposed a change to openstack/ironic-python-agent: Added metaclass statement  https://review.openstack.org/8687612:04
*** Alexei_987 has joined #openstack-ironic12:09
*** trapsig has quit IRC12:11
*** dkehn_ has joined #openstack-ironic12:22
*** davidlenwell_ has joined #openstack-ironic12:22
*** dkehnx has quit IRC12:24
*** davidlenwell has quit IRC12:24
*** blamar has joined #openstack-ironic12:34
*** lucas-hungry is now known as lucasagomes12:36
*** yuriyz has quit IRC12:39
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Use DiskPartitioner  https://review.openstack.org/8339912:41
jrollHaomeng: I see rameshg87 is no longer here, but fyi: we currently build the agent into a coreos ramdisk, check the imagebuild/coreos directory for instructions :) (this may change in the future)12:45
jrollmorning ironic :)12:45
agordeevjroll: morning!12:48
jrollheya agordeev :)12:48
Haomengjroll: morning, ok, thanks12:50
Haomengjroll: our new ramdisk support ipminode discovery?12:50
jrollwhat do you mean by ipminode discovery?12:51
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add ManagementInterface  https://review.openstack.org/8606312:52
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMITool to use the new ManagementInterface  https://review.openstack.org/8609212:52
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: SeaMicro to use the new ManagementInterface  https://review.openstack.org/8632812:52
Haomengjroll: means to auto-discovery the bm node with mac address auto-register12:52
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: IPMINative to use the new ManagementInterface  https://review.openstack.org/8658812:52
jrollHaomeng: not yet, but we'll be working on it12:53
Haomengjroll: ok12:53
Haomengjroll: great:)12:53
jrollright now the only info we need to add to ironic db is ipmi info and regular mac address12:53
Haomengjroll: yes12:53
jrolleverything else is auto-discover12:53
Haomengjroll: ok, got:)12:55
*** athomas has joined #openstack-ironic12:57
*** ndipanov has quit IRC12:58
*** matty_dubs|gone is now known as matty_dubs12:59
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add agent fields to virt driver  https://review.openstack.org/8513112:59
NobodyCamgood morning Ironic13:04
*** ndipanov has joined #openstack-ironic13:05
jrollmorning NobodyCam :)13:08
agordeevNobodyCam: morning13:08
agordeevHaomeng: morning :)13:08
NobodyCam:) good morning agordeev and Haomeng :)13:08
Haomengmorning, NobodyCam and agordeev :)13:09
*** trapsig has joined #openstack-ironic13:09
NobodyCamahh 86639 still only has 1 +2 :(13:11
*** dkehn_ is now known as dkehnx13:15
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479513:16
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Version agent lookup payloads  https://review.openstack.org/8522813:19
HaomengNobodyCam: :)13:20
NobodyCamTGIF!13:21
openstackgerritAleksandr Gordeev proposed a change to openstack/ironic-python-agent: Add FlowExtension  https://review.openstack.org/8541213:21
openstackgerritAleksandr Gordeev proposed a change to openstack/ironic-python-agent: Resolve circular import, introduce ExecuteCommandMixin  https://review.openstack.org/8634113:21
NobodyCamall ready for the weekend Haomeng ?13:21
HaomengNobodyCam: yes, my fri's evening:)13:22
NobodyCam:)13:22
HaomengNobodyCam: :)13:23
NobodyCamoh I almost forgot... Congratz Haomeng :)13:24
HaomengNobodyCam: thank you:)13:24
NobodyCamhehehehe :)13:24
agordeevtrapsig: ping13:24
HaomengNobodyCam: I am in vacations previous days, so get the good news from yesterday:)13:25
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Get rid of the swap partition  https://review.openstack.org/8372613:25
HaomengNobodyCam: :)13:25
NobodyCammorning lucasagomes :)13:25
lucasagomesNobodyCam, yo, morning!13:25
lucasagomesmorning Haomeng agordeev jroll13:25
NobodyCam:)13:25
lucasagomes:)13:25
Haomengmorning lucasagomes :)13:26
agordeevjroll: could you know what was happened wrong with jenkins for IPA?13:26
agordeevlucasagomes: also morning :)13:26
jrollagordeev: hm? where?13:27
NobodyCammorning jroll :)13:28
jrollgood afternoon lucasagomes :)13:28
jrollhey NobodyCam :)13:28
jrollit must be friday13:28
jrollso many :)13:28
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add version 1 of agent payload  https://review.openstack.org/8617313:28
jrollanyone know how to change a review topic without any new changes?13:30
agordeevjroll: it doesn't start check jobs for a few hours. Before today it was passing these checks extremely fast at most in few minutes13:31
jrollagordeev: maybe something wrong with infra, idk13:31
jrollif you have an example, I could look13:31
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add version 1 of agent payload  https://review.openstack.org/8617313:31
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Version agent lookup payloads  https://review.openstack.org/8522813:31
agordeevjroll: just take a look at my and vkozhukalov's patchsets for today. https://review.openstack.org/#/c/86163/ took about 2 hours. Is it normal?13:34
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add lookup method to agent vendor mappings  https://review.openstack.org/8640213:34
*** lazy_prince has quit IRC13:35
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Bring agent deploy driver up to date with virt  https://review.openstack.org/8649013:35
jrollagordeev: yeah, looks like an infra problem. the tests ran fast, just started with a delay13:36
jrollsee http://status.openstack.org/zuul/13:36
jrolleverything looks behind13:36
jrollthat will make today fun13:36
*** jbjohnso_ has joined #openstack-ironic13:40
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add configdrive generation to Nova driver  https://review.openstack.org/8619213:40
jrollside note: cores, I have all of the agent-related reviews up-to-date and rebased. most also have a bp/agent-driver topic :)13:42
*** linggao has joined #openstack-ironic13:53
*** trapsig has quit IRC14:02
lucasagomesdtantsur, the thing about the lambda in the patch decorator is pretty neat!14:03
lucasagomesdtantsur, cheers for that14:03
dtantsurlucasagomes, I'm glad that helps :)14:03
jbjohnso_so, is there any particular request on extending the authentication mechanism in confluent?  I see today the serial console is pretty much wide open in the shellinabox case14:05
jbjohnso_I was thinking about adding a method to request a console token that is specific to just one console rather than requiring something to create a lot of tenants to separate one node from another14:06
*** trapsig has joined #openstack-ironic14:07
*** russellb is now known as rustlebee14:07
*** overlayer has quit IRC14:11
*** ndipanov has quit IRC14:16
*** ndipanov has joined #openstack-ironic14:17
*** ndipanov is now known as engeepanoff14:17
NobodyCamwoo hoo14:18
NobodyCam86639 landed14:18
*** Haomeng|2 has joined #openstack-ironic14:19
*** Haomeng has quit IRC14:20
lucasagomesNobodyCam, w00t!14:25
lucasagomesdebug is working now?14:25
openstackgerritChris Krelle proposed a change to openstack/ironic: Fix for tripleO undercloud gate tests DO NOT MERGE  https://review.openstack.org/8552914:26
openstackgerritChris Krelle proposed a change to openstack/ironic: Workaround race with Neutron port updates  https://review.openstack.org/8436114:26
NobodyCamwe should get logging this time14:26
*** max_lobur has quit IRC14:26
*** max_lobur has joined #openstack-ironic14:27
NobodyCamwalkies time... brb14:32
linggaoDoes anyone know how the keystone part of ironic.conf.sample file is pulled in?14:35
linggaoI rebuilt my venv and still get old keystone section in the ironic.conf.sample14:36
linggaoThere is a python-keystoneclient>=0.7.0 in the requirements.txt14:37
linggaoand tox -evenv -- echo 'done' installs keystoneclient files under .tox/venv/lib/python2.7/site-packages14:38
linggaobut looks like it did not get latest files.14:39
linggaowhere should I point my repository to?14:39
lucasagomeslinggao, hmm oddly, you can:  source .tox/venv/bin/activate ;; pip install -r requirements.txt -U14:40
lucasagomesbut if you built from scratch I don't think you would need to do that ^14:40
linggaolucasagomes, good morning :). I just built from scratch.14:41
lucasagomesyea14:41
lucasagomesthat's pretty odd14:41
linggaodoes it has anything to do with the repository for the node?14:41
linggaoapt-cache policy python-keystoneclient14:41
linggaopython-keystoneclient:14:41
linggao  Installed: (none)14:41
linggao  Candidate: 2012.1-0ubuntu114:41
linggao  Version table:14:41
linggao     2012.1-0ubuntu1 014:41
linggao        500 http://us.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages14:41
linggaomaybe that's where the keystoneclient come from in the venv?14:42
lucasagomesit install it from pip14:42
lucasagomesdid you remove the .tox folder after rebuilding it?14:43
* lucasagomes 1 sec brb14:43
linggaoI created a new direcoty and totally built from scrach following the "Developer Quick-Start" instructions.14:45
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Accept new parameters for `prepare_image`  https://review.openstack.org/8672314:48
linggaolucasagomes, when I check .tox/venv/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py14:49
linggao file, it is different from the master version on the web.14:49
NobodyCamlinggao: does keystone need to cut a new release?14:51
NobodyCamclients come from pypi, don't they?14:52
linggaoNobodyCam, either that or something wrong with my env.14:53
linggaoNobodyCam, could you vi .tox/venv/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py  in your venv?14:53
linggaocheck line 27014:54
linggaoDoes it say help='Optionally specify a list of memcached server(s) to'  ?14:54
NobodyCamline 270: ' Acceptable values are MAC or ENCRYPT.  If MAC, token data is'14:56
linggaothat's old file. I have that too.14:58
NobodyCamhttps://pypi.python.org/pypi/python-keystoneclient/0.7.114:59
NobodyCamlast update 2014-03-2714:59
lucasagomesback15:00
NobodyCamwb lucasagomes15:00
lucasagomes:)15:00
linggaoAnd seems the Jekins build env points to a newer version of keystone.15:01
*** ifarkas_ has joined #openstack-ironic15:01
linggaoI mean keystoneclient15:01
linggaothat's why it complians that my sample file is not up to date. That's my theory. :)15:02
NobodyCamhumm15:02
linggaoso NobodyCam, you are right, the keystone needs to cut a new release.15:03
*** ifarkas has quit IRC15:03
linggaoor I need to get the latest keystoneclinet into my venv somehow.15:04
openstackgerritPablo Fernando Cargnelutti proposed a change to openstack/ironic: Overwriting node_is_available in IronicDriver  https://review.openstack.org/8421415:05
lucasagomeslinggao, try doing a pip install -U "python-keystoneclient>=0.7.0" to see if it downloads the last version15:07
linggaolucasagomes, nop. It is still the old one.15:09
lucasagomeslinggao, ouch15:09
openstackgerritAleksandr Gordeev proposed a change to openstack/ironic-python-agent: Improve six usage for builtin open function in tests  https://review.openstack.org/8693215:09
NobodyCamlinggao: git clone the latest repo source the ironic venv then pip install -u </path/to/cloned/KS Client/15:11
linggaoNobodyCam, I did that too. I rebuilt the whole thing from sratch this morning.15:14
linggaois there a way to check the version of keystonecient using pip?15:15
devanandamorning, all15:15
NobodyCampip freeze15:16
NobodyCamgood morning devananda15:16
agordeevdevananda: morning!15:16
linggaomorning devananda.15:16
*** ifarkas_ has quit IRC15:17
linggaoNobodyCam, pip show shows Version: 0.7.115:17
linggaowhich is identical to https://pypi.python.org/pypi/python-keystoneclient/15:17
linggaoSo I think the Jekins is ahead of us.15:18
lucasagomesmorning devananda15:18
dtantsurdevananda, morning!15:18
NobodyCammoring dtantsur :)15:18
NobodyCamlinggao: clone the KS client repo15:19
dtantsurNobodyCam, morning :)15:19
NobodyCamsource the ironic venv15:20
Mikhail_D_wkdevananda: Morning! :)15:20
NobodyCamthen pip install /path/to cloned/ repo15:20
NobodyCamthen deactavate15:20
NobodyCamthat should get the latest into your venv15:21
linggaoNoodayCam, the file that makes the trouble is  ironic/.tox/venv/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py15:21
linggaoclone keystoneclient will not put the correct files there.15:21
NobodyCamhumm15:22
NobodyCammaybe worth a question in keystone channel15:22
linggaoNobodyCam, or Jekens build env.15:22
linggaojust wonder how Jekens build env pull the keystoneclient files. Does it get it from pip or the git  master15:23
devanandalucasagomes: just replied to https://review.openstack.org/#/c/86722/15:25
NobodyCamshould be pip installing .. but maybe ask -infra15:25
* lucasagomes looks15:26
NobodyCamlinggao: do you have a link to the errors your getting15:26
*** zul has quit IRC15:26
* lucasagomes facepalm15:27
lucasagomesdevananda, yeah forgot that detail15:27
lucasagomeschanging my vote15:27
devanandalucasagomes: :)15:27
lucasagomesgood stuff porting it15:27
devanandalucasagomes: thanks15:28
devanandai think taht closes all the rc bugs we've identified15:28
linggaoNobodyCam, yes. https://review.openstack.org/#/c/64100/ .  I did not get any error in my env, the Jekins complians that the ironic.conf.sample file is not up to date.15:28
devanandalucasagomes: have you seen any more lately? if not - i'll ask ttx to cut RC2 as soon as this merges15:28
lucasagomesdevananda, none that I think that needs to be backported15:28
devanandaawesome15:28
linggaoNobodyCam http://logs.openstack.org/00/64100/16/check/gate-ironic-pep8/6b3af76/console.html15:29
*** ifarkas has joined #openstack-ironic15:30
*** zul has joined #openstack-ironic15:30
linggaoNobodyCam, basically I got the keystoneclient from pip, but Jekins got it from git master.15:30
* devananda hops on a phone call15:31
NobodyCamdevananda: it works better if you hold the phone to your ear :-p15:31
Shrewsdevananda: wouldn't it be better to be sitting still while on the phone?15:31
NobodyCamoh its def a Friday15:32
Shrews\o/15:32
linggaoShrews, I know someone who always walk around while on conf calls. He uses bluetooth :-). I think that's a good way to do exercise while work.15:33
Shrews:)15:34
linggaoNobodyCam, for now I'll manually copy the latest keystoneclient middleware files into my venv just to make Jekins happy about the sample file my venv creates.15:36
NobodyCamlinggao: that could cause issues in our gate15:38
linggaoNobodyCan, or we can ask keystone to release a new version of python-keystoneclient?15:39
linggaocurrent version was created on march 27th. And there are more changed in the conf section of keystoneclient/middleware/auth_token.py file.15:42
linggaoafter that date.15:42
NobodyCamlinggao: I have a change to conf.sample up with 85529 lets see what I get from the check15:43
NobodyCamthou seems like zuul is not doing much this morning15:46
linggaohmm, NobodyCam, I just checked, your sample file is ok.15:48
linggaoNobodyCam, just wonder how you get the latest keystoneclient from your venv.15:49
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Better check for available nodes  https://review.openstack.org/8468715:50
NobodyCamlinggao: all I did was git clone the ironic repo, and run tox -e py2715:51
NobodyCamoh there is now over 230 jobs in the queue15:53
*** romcheg has quit IRC15:57
*** eghobo has joined #openstack-ironic15:57
*** eghobo has quit IRC15:57
*** eghobo has joined #openstack-ironic15:58
*** romcheg has joined #openstack-ironic15:59
ShrewsSo, currently in the ironic nova driver, if we exceed the max client retries, we don't raise an error: https://github.com/openstack/ironic/blob/master/ironic/nova/virt/ironic/driver.py#L67115:59
ShrewsThat's different behavior from other areas. Is that something we should keep? Or a mistake?16:00
Shrewslucasagomes, devananda, NobodyCam: ^^^16:00
openstackgerritA change was merged to openstack/ironic-python-agent: Implement version 2 payload with more hw info  https://review.openstack.org/8647516:01
devanandaShrews: seems ilke an error16:01
*** ifarkas has quit IRC16:01
lucasagomesyeah looks inconsistent with the plug_vif as well16:02
lucasagomesif it can't plug a vif it raises16:02
NobodyCamya16:02
lucasagomesif it can't it just log heh16:02
Shrewsok, great. just wanted to verify16:02
NobodyCamlucasagomes: TY for the tests :)16:04
lucasagomesNobodyCam, yw :)16:04
*** romcheg has quit IRC16:10
*** trapsig has quit IRC16:11
*** matty_dubs is now known as matty_dubs|lunch16:18
*** comstud is now known as bearhands16:22
*** harlowja_away is now known as harlowja16:25
lifelessdevananda: new fun failure for you16:28
lifelessdevananda: I'll file this in a bug, but it will affect ironic too - http://paste.openstack.org/show/75578/16:28
*** martyntaylor1 has left #openstack-ironic16:30
*** engeepanoff has quit IRC16:30
devanandalifeless: ooh, that's interesting16:30
devanandalifeless: someone, forgetting who right now, pointed out that ResourceManager is not actually doing its job right now16:31
devanandai wasn't totally convinced of it, but it seems related16:31
lucasagomes:(16:33
lucasagomeswe might start trying to incorporate some fixes into nova to adjust the core code to understand a bit more ironic16:34
lucasagomeswe have been working around it now (adding a new compute manager etc...)16:35
lucasagomesthat ^ might result in another workaround16:35
devanandayea16:36
devanandai have a couple sessions proposed around some of those things16:36
devanandaeg, (host, node) is a pain for everyone16:36
devanandacauses race conditions when running >1 n-cpu with ironic16:37
lucasagomesyeah16:37
lucasagomesthat's def something we need to architect better in nova16:37
lucasagomesusing N cpu with same hostname16:38
lucasagomesand stubing the init_host thing16:38
devanandaright16:38
devanandaanother work around16:38
lucasagomesyeah16:38
openstackgerritA change was merged to openstack/ironic: Do not sync power state for new invalidated nodes  https://review.openstack.org/8652916:38
lucasagomeswell, we can talk about it in the summit16:38
openstackgerritlinggao proposed a change to openstack/ironic: Support serial console access  https://review.openstack.org/6410016:39
lucasagomesnew PTL for nova, so I think it will be a good way to start16:39
jrolllucasagomes: hey, added tests for you here :) https://review.openstack.org/#/c/85131/16:39
NobodyCamoh speaking of PTL's Congratz devananda :)16:39
lucasagomesjroll, oh cheers, will take a look16:39
lucasagomesheh true, congrats for the nomination devananda16:39
devanandalol16:40
devanandathx16:40
devanandalucasagomes: re: whitelist / enabled drivers ... i see https://review.openstack.org/#/c/85996/ just landed. i'll update my patch to use that option16:40
devanandas/just//16:41
lucasagomesdevananda, yeah I thought about it, use drivers_whitelist then16:41
lucasagomesso we don't need to change devstack yet again16:41
devanandalucasagomes: fwiw, i like "enabled_drivers" as an option name better :p16:41
lucasagomesdevananda, and sorry for stepping on ur toes, I totally forgot that patch you had16:41
devanandanp16:41
lucasagomesdevananda, :P I gotta say, I do as well16:41
lucasagomeshah16:41
devanandaheh16:41
*** athomas has quit IRC16:44
*** vkozhukalov has quit IRC16:54
devanandalucasagomes: for future reference, this could have also been done in stages by adding "fake" to the conf in ironic until devstack changed16:57
lucasagomesdevananda, yeah, that would work as well... you wanna do it to keep the enabled_drivers name?16:58
devanandai think it's a more consistent option name16:59
lucasagomesdevananda, +116:59
lucasagomesjust add a TODO to remove it once it's added to devstack16:59
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Accept new parameters for `prepare_image`  https://review.openstack.org/8672317:01
*** matty_dubs|lunch is now known as matty_dubs17:02
lucasagomessomeone said that there are only two hard things in computer science: cache invalidation and naming things17:02
*** jdob_ has joined #openstack-ironic17:02
devanandahah17:02
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Better handling of missing drivers  https://review.openstack.org/8357217:02
bearhandsyeah, and I'm not sure which one is harder17:03
lucasagomesnaming17:03
JoshNangand off by one errors :)17:03
lucasagomes:P17:03
lucasagomesyeah17:03
*** jdob has quit IRC17:03
*** Mikhail_D_ltp has joined #openstack-ironic17:06
*** Haomeng has joined #openstack-ironic17:07
*** Haomeng|2 has quit IRC17:07
*** rwsu has quit IRC17:09
*** jdob_ is now known as jdob17:09
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Accept new parameters for `prepare_image`  https://review.openstack.org/8672317:11
devanandagotta run, bbiah or so17:12
NobodyCamok enjoy devananda17:12
*** jdob has joined #openstack-ironic17:12
*** jdob_ has joined #openstack-ironic17:16
*** jdob has quit IRC17:16
*** overlayer has joined #openstack-ironic17:16
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Accept new parameters for `prepare_image`  https://review.openstack.org/8672317:17
*** newell_ has joined #openstack-ironic17:22
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add DriverFieldsFactory  https://review.openstack.org/8697117:43
lucasagomesNobodyCam, ^ jroll (it touches the part you just changed to the agent)17:44
lucasagomesit's WIP anyway, lemme know17:44
*** hemna has quit IRC17:45
NobodyCamlucasagomes: ack , looking17:45
lucasagomes:)17:46
lucasagomesI'm done for today, g'night everyone enjoy the weekend!17:47
NobodyCamhave a great weekend lucasagomes17:47
lucasagomesNobodyCam, you too! have a pint for me17:47
*** lucasagomes is now known as lucas-dinne17:47
NobodyCamlol it just about noon here... so about 5 hours17:47
*** lucas-dinne is now known as lucas-dinner17:47
NobodyCamthen alien ale!17:48
lucas-dinnerNobodyCam, heh 5 hours will go quick!17:48
lucas-dinneryeah17:48
NobodyCam:)17:48
lucas-dinnerit's already beer o'clock for me17:48
lucas-dinner:D17:48
jrollsee ya lucas-dinner :)17:48
NobodyCamenjoy lucas-dinner17:48
lucas-dinnerjroll, see ya17:48
* lucas-dinner runs17:48
NobodyCamfor me it's bagel time...brb17:48
jrolllucas-dinner: at a glance +1, I like where this is going17:49
*** romcheg has joined #openstack-ironic17:49
*** Haomeng|2 has joined #openstack-ironic17:52
*** Haomeng has quit IRC17:53
*** ilives has quit IRC17:58
openstackgerritRussell Haering proposed a change to openstack/ironic: Drivers may expose a top-level passthru API  https://review.openstack.org/8191918:01
*** Alexei_987 has quit IRC18:02
*** Haomeng|2 has quit IRC18:04
ShrewsNobodyCam: mmm, alien ale....18:05
*** Haomeng has joined #openstack-ironic18:05
NobodyCam:)18:06
* devananda shakes his head at launchpad timeout errors18:07
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Support for v2 agent payloads  https://review.openstack.org/8698418:16
JayFooh thanks JoshNang18:16
linggaodevananda, ping18:16
JoshNangJayF: no prob. except a rebase made it into this commit. grr.18:16
NobodyCamwb devananda18:16
devanandalinggao: pong18:17
JayFyeah I was about to ask about that, looked like it had some stuff in it unrelated18:17
linggaodevananda, the API v1/nodes/<uuid>/states/console returns one string,18:17
linggaobut the command line output cannot handle a string, it has to be a dic18:18
* NobodyCam steps aways for a few minutes18:18
linggaodict18:18
linggaodevananda, so ironic node-get-console returns this: 'unicode' object has no attribute 'iteritems'18:19
devanandalinggao: sounds like a bug18:21
openstackgerritJim Rollenhagen proposed a change to openstack/ironic-python-agent: Use configurable driver name in lookup URL  https://review.openstack.org/8620118:22
linggaodevananda, should we change api to return ['consle': url] or should we fix the command line to handle string output?18:22
devanandalinggao: the API should return a JSON object in the HTTP response body18:22
linggaodevananda, ok. I'll go ahead fix the API part.18:23
*** max_lobur has quit IRC18:23
devanandalinggao: what information must be exposed in the API for a console connection? is it just a single URL?18:23
linggaoyes, just a url.18:23
devanandalinggao: yea, please file a bug. I think the fix is to add18:24
devanandaclass NodeConsole(base.APIBase)18:25
linggaodevananda, the nova get-rdp-console returns this:18:25
linggao<console>18:25
linggao  <type>rdp-html5</type>18:25
linggao  <url>http://example.com:6083/?token=f9906a48-b71e-4f18-baca-c987da3ebdb3</url>18:25
linggao</console>18:25
devanandato ironic/api/controllers/v1/node.py18:25
devanandaright. type. that's what I thought18:25
devanandain addition to URL, we need to specify what protocol / type of console it is18:25
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Add version 1 of agent payload  https://review.openstack.org/8617318:25
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Support for v2 agent payloads  https://review.openstack.org/8698418:25
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479518:25
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Version agent lookup payloads  https://review.openstack.org/8522818:25
linggaodevananda, so the type is shellinabox in our case?18:26
devanandalinggao: hmm, that's the softawre we're using, not the protocol18:27
devanandardp-html5 is a protocol18:27
devanandai'm not sure off hand what shellinabox is using. probably rdp?18:27
devanandaSo, i tlooks like all the MP fixes landed18:28
devanandaanyone around want to give that a test before we cut RC2?18:28
devanandaNobodyCam? adam_g ?18:28
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Support for v2 agent payloads  https://review.openstack.org/8698418:29
linggaonova also has get-vnc-console the type is 'novnc'18:29
adam_gdevananda, sorry, MP?18:29
devanandamilestone proposed18:30
adam_goh :)18:30
devanandai backported 4 fixes to the icehouse milestone proposed branch18:30
NobodyCamdevananda: my env is trashed atm18:30
adam_gdevananda, yeah i can give it a quick test in devstack right now18:30
devanandaall worked for me, but getting a couple more folks runnin git before ttx tags another RC would be great18:30
devanandaadam_g: thanks!18:31
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479518:31
adam_gthe baremetal scenario testing is in tempest and is running in our gate.. but not passing. trying to figure out why now18:31
devanandaadam_g: anything I can help unblock?18:31
adam_gdevananda, kinda scratching my head right now. it times out waiting on provisioning. wondering if 300 sec. timeout is not large enough for the nodes that the gate uses. trying now on a similar rackspace instance18:32
devanandaadam_g: yea. try 10x that and see18:33
devananda300s is less than hw POST time, fwiw18:33
adam_gdevananda, yeah, im gonna put up a devstack patch that does that to see if it helps in the experimental  check18:33
devanandaso that will probably fail on bare metal18:33
*** jistr has quit IRC18:36
adam_gdevananda, also an error here that i haven't seen before http://logs.openstack.org/92/86092/7/check/check-tempest-dsvm-virtual-ironic/623504d/logs/screen-ir-cond.txt.gz18:37
*** jdob_ is now known as jdob18:38
devanandaadam_g: timestamp?18:39
adam_gdevananda, hmm maybe i gave you the wrong long. one sec18:39
adam_ghttps://review.openstack.org/8699018:39
adam_gdevananda, http://logs.openstack.org/92/86092/7/check/check-tempest-dsvm-virtual-ironic/623504d/logs/screen-ir-cond.txt.gz18:40
adam_g2014-04-11 14:15:35.851 20070 TRACE ironic.openstack.common.rpc.amqp MACAlreadyExists: A Port with MAC address fa:16:3e:2c:bf:ce already exists.18:40
ShrewsCan anyone confirm that pressing the "Rebase Change" button in gerrit will re-add existing votes? I can never remember18:41
devanandaadam_g: that seems to be the result of an intentional negative test to add a dup MAC18:42
devanandaShrews: yes18:42
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Encapsulate Ironic client retry logic  https://review.openstack.org/8310518:42
devanandaShrews: even if you rebase locally, it still does that, as long as nothing in the patchset changes18:42
adam_gdevananda, ah! from the API tests, i assume?18:43
devanandaadam_g: yep18:43
devanandaadam_g: i've seen that even in tests that pass18:43
adam_ggood to know18:43
NobodyCamdevananda: lifeless: I may be crazy, blind or both. but can not see where check-tripleo-ironic-undercloud-precise is applying the patch set being tested. should not DIB_REPOREF_ironic be getting set?18:48
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Complete wrapping ironic client calls  https://review.openstack.org/8699318:49
devanandaNobodyCam: afaik yes, but I haven't been testing wth tripleo in a bit18:53
*** Haomeng has quit IRC18:55
NobodyCam:) digging in now18:56
*** Haomeng has joined #openstack-ironic18:56
*** eghobo has quit IRC18:59
* devananda lunches19:02
NobodyCamnm it is... just in a different log file19:04
*** k4n0 has joined #openstack-ironic19:07
k4n0Hello folks19:13
*** max_lobur has joined #openstack-ironic19:13
NobodyCammorning k4n019:14
k4n0:) Its late night in India19:14
k4n0but good morning to you19:14
NobodyCamits afternoon here in roswell19:14
k4n0Good afternoon :)19:14
k4n0Actually I am trying to get some eyes on this problem i am facing http://lists.openstack.org/pipermail/openstack-dev/2014-April/032481.html19:15
NobodyCamhehehe19:15
NobodyCamhummm19:16
NobodyCamdo you have console access? have you set troubleshoot and manually checked to see that the node is reporting?19:17
k4n0I have serial console access, how do i set troubleshoot? sorry, i dont know much about diskimage-builder19:18
k4n0node is reporting fine, i can see the ironic callback happening19:18
k4n0the callback fails19:18
adam_gdevananda, a quick run tempest thru  against milestone-proposed works fine in my environment. not sure there's anything specific you want tested19:21
*** rwsu has joined #openstack-ironic19:24
k4n0Any other steps i can take to isolate this problem?19:26
NobodyCamk4n0: looking at some things now19:27
k4n0NobodyCam: cool19:27
NobodyCamhumm19:33
NobodyCamk4n0: ok19:36
NobodyCamadd a kernel param.. https://github.com/openstack/diskimage-builder/blob/master/elements/ramdisk/extra-data.d/scripts/init#L5019:36
k4n0NobodyCam: i will add troubleshoot ?19:37
NobodyCamhttps://github.com/openstack/ironic/blob/master/etc/ironic/ironic.conf.sample#L84319:37
NobodyCam:)19:37
NobodyCamthat should drop teh node to a bash shell when it hits the error19:38
k4n0NobodyCam: ok, i will add "troubleshoot" to the pxe_append_params conf19:38
k4n0NobodyCam: I will try this out in morning, will let you know my findings.19:39
NobodyCamhttps://github.com/openstack/diskimage-builder/blob/master/elements/ramdisk/extra-data.d/scripts/d/init-func#L23219:39
NobodyCamlooks like you'll need to add troubleshoot=119:40
k4n0got it19:40
NobodyCam:)19:40
k4n0NobodyCam: what should i do after i get the shell? cause i know the disk is not being detected19:41
NobodyCamsee if you can figure out whats up..19:41
NobodyCamis it missing a driver? is comming in as sdb19:41
NobodyCamhard to say what to try not knowing whats wrong19:42
NobodyCamsee if dmesg / syslog has any insights19:42
k4n0might be a driver, i will check, its not sdb for sure, i have checked /proc/diskstats19:42
k4n0sure, i will dig in19:42
k4n0thanks for the help :)19:42
NobodyCam:)19:42
k4n0Cya, good night19:43
NobodyCamyou too :)19:43
*** k4n0 has left #openstack-ironic19:43
NobodyCamlinggao: just fyi .. my conf.sample passed19:45
linggaoNobodyCam, good. Mine sample looks promising now.  Not sure why it started working. I must messed up somehting before :)19:47
NobodyCam:)19:47
*** vkozhukalov has joined #openstack-ironic19:59
*** overlayer has quit IRC20:01
*** max_lobur1 has joined #openstack-ironic20:04
*** max_lobur has quit IRC20:04
openstackgerritChris Krelle proposed a change to openstack/ironic: Fix for tripleO undercloud gate tests DO NOT MERGE  https://review.openstack.org/8552920:07
*** romcheg has quit IRC20:11
*** romcheg has joined #openstack-ironic20:13
*** overlayer has joined #openstack-ironic20:15
*** overlayer has quit IRC20:16
*** romcheg has quit IRC20:22
devanandaadam_g: great, thanks! I haven't been able to reproduce the bugs whose fixes i backported, so I'm gonna call it good at this point20:22
adam_gcool20:23
*** rustlebee is now known as russellb20:27
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Add discrete states for event log status sensors  https://review.openstack.org/8702320:38
*** Haomeng has quit IRC20:44
*** Haomeng has joined #openstack-ironic20:46
*** rwsu has quit IRC20:46
*** rwsu has joined #openstack-ironic20:46
jbjohnso_wow, that's a long check queue20:50
NobodyCamoh ya20:51
openstackgerritJarrod Johnson proposed a change to stackforge/pyghmi: Gracefully flag unrecognized discrete sensor states  https://review.openstack.org/8702720:51
*** harlowja is now known as harlowja_away20:51
JayFIf you look at the graphs on status.openstack.org/zuul20:51
JayFit looks like we went down to 0 workers, got back up again and started chewing through a backlog, and now it's back down to 0 workers20:51
NobodyCam:)20:52
*** jdob has quit IRC20:53
devanandaJayF: infra's capacity was cut in half for a day, essentially. still playing catchup20:56
*** romcheg has joined #openstack-ironic20:56
NobodyCamI have some only hardware I could give them! :-p20:57
NobodyCamsonly/old/20:57
*** harlowja_away is now known as harlowja21:00
devanandabbiab21:00
*** jrist has quit IRC21:00
NobodyCamok :)21:01
NobodyCamI may go look for some food21:01
*** max_lobur1 has quit IRC21:01
jbjohnso_well, maybe my pyghmi changes will get a verify by monday21:02
jbjohnso_it's a good day to go home and stop thinking about computers....21:02
*** jrist has joined #openstack-ironic21:02
NobodyCam:) lol queues only at 11 and 1/2 hours now :)21:03
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479521:05
*** jbjohnso_ has quit IRC21:07
*** linggao has quit IRC21:13
*** romcheg has quit IRC21:15
*** Mikhail_D_ltp has quit IRC21:16
*** matty_dubs is now known as matty_dubs|gone21:34
openstackgerritRussell Haering proposed a change to openstack/ironic: Fix conductor.manager test assertion order  https://review.openstack.org/8703521:39
*** stevehuang has quit IRC21:41
*** stillbert is now known as mikal21:45
*** romcheg has joined #openstack-ironic21:59
openstackgerritJosh Gachnang proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479522:00
JayFSo question, and it's especially relevant now with the zuul backlogged so badly22:04
JayFis it OK to +2 an unverified patch if it works and tests pass locally?22:04
JayFThere are some things I know to be good, and tests pass locally, and I'd like to get them in the queue for merging over the weekend22:05
openstackgerritChris Krelle proposed a change to openstack/ironic: Add Logging.  https://review.openstack.org/8512422:07
*** romcheg1 has joined #openstack-ironic22:07
NobodyCamJayF: is it a critical patch?22:07
JayFNobodyCam: ^ it looks like you had some bonus whitespace leak into manager.py on that patch22:08
NobodyCamnote it wont merge untill it ia tested22:08
NobodyCam:-p22:08
JayFNobodyCam: yeah, I know it won't be merged until tested, that's why I am curious if it's OK to +222:08
JayFand 'critical'... not really, but at this point in an attempt to get a prototype working we're using master + patches that are up for review22:08
JayFand I'm trying to do what I can on the IPA side to reduce our patchset and get things in master22:09
JayFand when there are 12h long jenkins queues, I tend to want to reserve a place in line before the weekend, hehe22:09
NobodyCamdoh.. darn tab key22:09
NobodyCamJayF: yes it is ok.. but as another reviewer please note that you are +2 with out jenkins results22:10
NobodyCam:-p22:10
*** romcheg has quit IRC22:11
openstackgerritChris Krelle proposed a change to openstack/ironic: Add Logging.  https://review.openstack.org/8512422:11
NobodyCamwhite space fix22:11
JayFI'll look again22:11
JayFalthough my votes don't matter ;)22:12
NobodyCamlol they DO matter22:12
JayFyour commit message says you're adding things at info logging level, but here -> https://review.openstack.org/#/c/85124/5/ironic/conductor/utils.py it's at audit level. Intentional?22:12
NobodyCamyes22:13
*** davidlenwell_ is now known as davidlenwell22:13
JayFI just reread your commit message22:13
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add version 1 of agent payload  https://review.openstack.org/8617322:13
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Support for v2 agent payloads  https://review.openstack.org/8698422:13
NobodyCamdd AUDIT level logging to conductor: utils/manager for power22:13
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Adding a reference driver for the agent  https://review.openstack.org/8479522:13
NobodyCamoperations. also adds INFO to sync power state in manager.22:13
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Version agent lookup payloads  https://review.openstack.org/8522822:13
NobodyCami addeed note about audit22:13
JayFcool, thanks. Logging is nice.22:13
JayFCaring about logging levels is /very/ nice22:14
JayFI've run software that logs everything at INFO, and I've run software that the best logs were strace and tcpdump :)22:14
JayFit's always nice to have thought put into log levelling and such22:14
NobodyCambeen there done that one.. wieshark is your friend22:15
*** romcheg1 has quit IRC22:15
NobodyCam*wire Shark22:15
NobodyCamyou mean I cann't just "PRINT" entering <class,function,module>22:16
NobodyCamhad that one too22:17
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add lookup method to agent vendor mappings  https://review.openstack.org/8640222:17
NobodyCamI recall one java listener that would log Gib of junk but you had to keep > 5 hours of log. if you wanted to be able to get to the root cause of a failure :-p22:19
* NobodyCam is SO happy to not work for Java shop any more22:19
*** blamar has quit IRC22:20
openstackgerritRussell Haering proposed a change to openstack/ironic: Drivers may expose a top-level passthru API  https://review.openstack.org/8191922:21
*** eghobo has joined #openstack-ironic22:22
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Enabling translation through IPA to begin  https://review.openstack.org/8704722:23
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add lookup method to agent vendor mappings  https://review.openstack.org/8640222:25
openstackgerritJim Rollenhagen proposed a change to openstack/ironic: Add lookup method to agent vendor mappings  https://review.openstack.org/8640222:31
NobodyCamjroll: lol "Does exactly that."22:32
jrollI mean...22:32
NobodyCamhehe :)22:33
*** overlayer has joined #openstack-ironic22:33
*** overlayer has quit IRC22:33
jroll:P22:33
*** stack_ has quit IRC22:34
openstackgerritJosh Gachnang proposed a change to openstack/ironic-python-agent: Enabling translation for IPA to begin  https://review.openstack.org/8704722:39
*** max_lobur has joined #openstack-ironic22:51
*** epim has joined #openstack-ironic22:53
*** epim has quit IRC22:59
*** epim has joined #openstack-ironic23:11
*** zdiN0bot has joined #openstack-ironic23:11
NobodyCamok all have a good weekend.. I'll be back to check on tests23:19
devanandaJoshNang: hi! why do translation in IPA?23:26
*** epim has quit IRC23:29
devanandaheading out in a bit... ya'll have a good weekend~!23:29
NobodyCamyou too!23:29
*** Haomeng has quit IRC23:29
* devananda might be back online later ... need to swap harddrive into a new laptop and related stuff23:29
NobodyCamnew laptop???23:29
NobodyCamohh23:29
devanandasame model23:30
NobodyCamlol23:30
devanandamy current one seems to have a loose transistor23:30
NobodyCam:)23:30
devanandait randomly refuses to start23:30
NobodyCamnice23:30
devanandafix is: shake it23:30
devananda:(23:30
*** Haomeng has joined #openstack-ironic23:30
jrollheh23:31
jrollbetter than never boots :P23:31
JoshNangdevananda: we'll be sending errors and log messages back to ironic. where's the cut off for what should and shouldn't be translated?23:32
devanandaJoshNang: good question. things that are user-facing should probably be translated, but based on the Accept-Language in teh HTTP header23:32
devanandaJoshNang: so the agent <-> ironic communication doesn't make a whole lot of sense to translate23:32
devanandathe agent won't know what language the end user is using23:33
devanandaand the OS in which the agent runs probably won't have had it's locale configured, since it's a PXE'booted ramdisk23:33
devanandathere's evena large debate as to whether services (eg, ironic-conductor) should i18n their LOG messages23:33
devanandabecause google searching for a translated LOG message is ... unhelpful23:34
JoshNangheh i can imagine23:34
JoshNangthat was my main thought...translated log messages for operators23:34
devanandaif ironic-conductor sends accept-lang to the agent23:34
devanandai could see that23:35
devananda*that that would fucntion23:35
devanandabut i'm not sure it's useful23:35
JoshNangthat's fine with me23:35
devanandaanyhow, good things to think about. maybe raise on the i18n list? I need to run :)23:35
russell_hdevananda: have a good weekend23:35
JoshNangsounds good! have a good weekend!23:35
*** zdiN0bot has quit IRC23:39
*** lucas-dinner has quit IRC23:44
openstackgerritA change was merged to openstack/ironic-python-agent: Use # instead of """ for copyright blocks  https://review.openstack.org/8645523:47
openstackgerritRussell Haering proposed a change to openstack/ironic-python-agent: Organize agent extensions  https://review.openstack.org/8705723:50
*** ilives has joined #openstack-ironic23:54
*** eghobo has quit IRC23:56
*** pradipta` has joined #openstack-ironic23:57
*** ilives has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!