*** igordcard has joined #openstack-ironic | 00:04 | |
*** Masahiro has joined #openstack-ironic | 00:07 | |
*** alexpilotti has quit IRC | 00:08 | |
*** Masahiro has quit IRC | 00:11 | |
*** smoriya has joined #openstack-ironic | 00:19 | |
*** Masahiro has joined #openstack-ironic | 00:44 | |
*** ryanpetrello has joined #openstack-ironic | 00:44 | |
*** ryanpetrello has quit IRC | 00:51 | |
*** ryanpetrello has joined #openstack-ironic | 01:00 | |
*** igordcard has quit IRC | 01:15 | |
*** naohirot has joined #openstack-ironic | 01:19 | |
naohirot | good morning ironic | 01:20 |
---|---|---|
*** yuanying has joined #openstack-ironic | 01:30 | |
*** yuanying_ has quit IRC | 01:33 | |
*** ryanpetrello has quit IRC | 01:37 | |
*** chenglch has joined #openstack-ironic | 01:42 | |
*** Haomeng has joined #openstack-ironic | 01:54 | |
*** Masahiro has quit IRC | 02:01 | |
*** Masahiro has joined #openstack-ironic | 02:04 | |
*** ryanpetrello has joined #openstack-ironic | 02:10 | |
*** zyluo has joined #openstack-ironic | 02:38 | |
*** ramineni has joined #openstack-ironic | 02:41 | |
openstackgerrit | Tan Lin proposed openstack/ironic: Add AMT-PXE-Driver to deploy cloud on PC https://review.openstack.org/135184 | 02:42 |
*** ryanpetrello has quit IRC | 02:44 | |
*** ryanpetrello has joined #openstack-ironic | 02:51 | |
ramineni | JoshNang: hi, there? | 02:52 |
*** achanda has joined #openstack-ironic | 03:00 | |
*** achanda has quit IRC | 03:02 | |
*** Haomeng has quit IRC | 03:06 | |
*** Masahiro has quit IRC | 03:07 | |
*** chenglch has quit IRC | 03:16 | |
*** Haomeng has joined #openstack-ironic | 03:17 | |
*** zyluo has quit IRC | 03:19 | |
*** zyluo has joined #openstack-ironic | 03:21 | |
*** ryanpetrello has quit IRC | 03:23 | |
*** Masahiro has joined #openstack-ironic | 03:30 | |
*** david-lyle_afk has quit IRC | 03:31 | |
*** Masahiro has quit IRC | 03:35 | |
*** Masahiro has joined #openstack-ironic | 03:40 | |
*** david-lyle_afk has joined #openstack-ironic | 03:43 | |
*** Masahiro has quit IRC | 03:48 | |
*** david-lyle_afk has quit IRC | 03:50 | |
*** Masahiro has joined #openstack-ironic | 03:52 | |
*** bradjones has quit IRC | 03:53 | |
*** zyluo has quit IRC | 03:53 | |
*** zyluo has joined #openstack-ironic | 03:54 | |
*** david-lyle_afk has joined #openstack-ironic | 04:02 | |
*** david-lyle_afk has quit IRC | 04:02 | |
*** Masahiro has quit IRC | 04:02 | |
*** lazy_prince has quit IRC | 04:08 | |
*** lazy_prince has joined #openstack-ironic | 04:09 | |
*** Masahiro has joined #openstack-ironic | 04:09 | |
*** jerryz has joined #openstack-ironic | 04:13 | |
*** yuanying_ has joined #openstack-ironic | 04:15 | |
*** jmanko has joined #openstack-ironic | 04:17 | |
*** yuanying has quit IRC | 04:18 | |
*** jmankov has quit IRC | 04:20 | |
*** Marga_ has joined #openstack-ironic | 04:35 | |
*** david-lyle_afk has joined #openstack-ironic | 05:01 | |
*** rushiagr_away is now known as rushiagr | 05:07 | |
*** pensu has joined #openstack-ironic | 05:21 | |
*** Marga_ has quit IRC | 05:33 | |
*** Marga_ has joined #openstack-ironic | 05:35 | |
*** Masahiro has quit IRC | 05:38 | |
*** Masahiro has joined #openstack-ironic | 05:38 | |
openstackgerrit | Tan Lin proposed openstack/ironic-specs: Bare Metal Trust Using Intel TXT https://review.openstack.org/133902 | 05:48 |
*** enterprisedc has quit IRC | 05:54 | |
*** victor_lowther has quit IRC | 05:59 | |
*** k4n0 has joined #openstack-ironic | 06:01 | |
*** cuihaozhi has joined #openstack-ironic | 06:02 | |
*** victor_lowther has joined #openstack-ironic | 06:02 | |
cuihaozhi | hi, when i boot instance from nova, if i must create a subnet with neutron? | 06:03 |
cuihaozhi | because nova-compute.log : "Network requires port_security_enabled and subnet associated in order to apply security groups." | 06:04 |
Haomeng | cuihaozhi: hi | 06:12 |
cuihaozhi | Haomeng,hi | 06:12 |
Haomeng | cuihaozhi: yes, nova will select default subnet which is created by neutron | 06:13 |
Haomeng | cuihaozhi: neutron will select one of them | 06:14 |
cuihaozhi | Haomeng: thank you, i try it :) | 06:14 |
Haomeng | cuihaozhi: but if we have multi networks that are L2, we have to input one net-id as the argument for nova boot | 06:14 |
Haomeng | cuihaozhi: welcome | 06:14 |
cuihaozhi | Haomeng: ok, i just create 1 net and 1 subnet. test enviroment for ironic :DD | 06:15 |
Haomeng | cuihaozhi: ok | 06:16 |
Haomeng | cuihaozhi: you can take the devstack env as reference | 06:16 |
*** victor_lowther has quit IRC | 06:16 | |
Haomeng | cuihaozhi: that is good example | 06:16 |
Haomeng | cuihaozhi: but it use vm as bm | 06:16 |
Haomeng | cuihaozhi: if you are trying to boot bm with physical network, maybe our ironic support neutron flat net only | 06:17 |
*** victor_lowther has joined #openstack-ironic | 06:17 | |
cuihaozhi | Haomeng: ye, i use flat net follow the install-guide | 06:17 |
Haomeng | cuihaozhi: ok | 06:18 |
Haomeng | cuihaozhi: good luck | 06:18 |
*** Marga_ has quit IRC | 06:20 | |
cuihaozhi | Haomeng: if pxe server run succece,which process i can see? dnsmasq? | 06:23 |
Haomeng | cuihaozhi: yes, and check the pxe conf | 06:24 |
Haomeng | cuihaozhi: but it is better to monitor the server's console output | 06:24 |
Haomeng | cuihaozhi: or redirect the os serial dev to pyhsical serial, and we can run ipmitool to monitor the os console output via pyhsical serial over lan | 06:25 |
cuihaozhi | Haomeng: thank u very much. i will check them and run again. | 06:26 |
Haomeng | cuihaozhi: ok | 06:27 |
*** Masahiro has quit IRC | 06:29 | |
*** mrda is now known as mrda-away | 06:31 | |
*** Masahiro has joined #openstack-ironic | 06:31 | |
*** tylerdurden has joined #openstack-ironic | 06:32 | |
*** shakamunyi has quit IRC | 06:34 | |
*** chenglch has joined #openstack-ironic | 06:46 | |
cuihaozhi | Haomeng: sorry another question, dnsmasq interface(like:tapb77dd3e2-9e) should be in bridge br-int? is that right? | 07:16 |
cuihaozhi | Haomeng:i think need another phys-eth interface in same brige, right? | 07:17 |
Haomeng | cuihaozhi: did you run devstack to install the env? | 07:17 |
Haomeng | cuihaozhi: do you want to real pyhsical server? | 07:17 |
cuihaozhi | Haomeng: em, i haven't use devstack before | 07:18 |
cuihaozhi | Haomeng:i can control the physical server'power with ironic node-set-power-state now | 07:18 |
Haomeng | cuihaozhi: you want to use devstack env to control the physical server? | 07:19 |
Haomeng | cuihaozhi: devsstack will use vm as bm to run ironic provision | 07:19 |
Haomeng | cuihaozhi: if you run with real pyhsical server, try to follow this doc - http://docs.openstack.org/developer/ironic/deploy/install-guide.html | 07:20 |
Haomeng | cuihaozhi: to setup neutron flat net to connect to physical server | 07:20 |
cuihaozhi | Haomeng: em, yes i follow this doc. | 07:20 |
Haomeng | cuihaozhi: so if can not control server power by node-set-power-state, is your current issue, right? | 07:21 |
Haomeng | cuihaozhi: can you ping from ironic conductor to ipmi ip? | 07:21 |
Haomeng | cuihaozhi: did you run with ipmi_pxe driver? | 07:21 |
cuihaozhi | Haomeng:ipmi_pxe drver yes | 07:21 |
Haomeng | cuihaozhi: ok, ping the ipmi ip from conductor side to see if it works | 07:22 |
cuihaozhi | Haomeng: current issue is when physerver start,it can't find pxe server good.. | 07:22 |
Haomeng | cuihaozhi: make sure your ipmi port in switch is connected with same L2 for our ironic conductor physical network port | 07:22 |
Haomeng | cuihaozhi: dont worry, check step by step | 07:23 |
Haomeng | cuihaozhi: ping ipmi node first to make sure ipmi works | 07:23 |
cuihaozhi | Haomeng: thank u,i will check bridge again. | 07:23 |
Haomeng | cuihaozhi: then reboot server and tcpdump the dhcp ra packages from conductor physical network ports | 07:24 |
cuihaozhi | Haomeng:ok i check it :)) | 07:24 |
Haomeng | cuihaozhi: to see if the physical server's dhcp request can be sent to conductor's physical nic | 07:24 |
Haomeng | cuihaozhi: then to check dnsmasq binding interfaces | 07:25 |
Haomeng | cuihaozhi: to make sure the interface is connected to the physical nic which geting the dhcp request from bm side | 07:25 |
Haomeng | cuihaozhi: ok | 07:25 |
cuihaozhi | Haomeng: my all servers in same host, is this problem? | 07:25 |
cuihaozhi | Haomeng:all openstack service | 07:25 |
Haomeng | cuihaozhi: no | 07:25 |
Haomeng | cuihaozhi: that is devstack default env I think | 07:26 |
cuihaozhi | Haomeng: ok. | 07:26 |
Haomeng | cuihaozhi: but make sure nova conf is configured to use our ironic driver | 07:26 |
Haomeng | cuihaozhi: for such all-in-one env, have to change nova.conf, and our ironic will be this compute node with ironic hypervisor type | 07:27 |
cuihaozhi | Haomeng: em, i find ovs-brige is different with install-guide. | 07:28 |
cuihaozhi | Haomeng: port like "int-br-eth2" not auto added in bridge br-int such as. | 07:28 |
cuihaozhi | Haomeng: then i added it myself and try again. | 07:29 |
Haomeng | cuihaozhi: in guide there is - ovs-vsctl add-port br-eth2 eth2 | 07:30 |
Haomeng | did you "restarting the Neutron Open vSwitch agent," | 07:30 |
Haomeng | cuihaozhi: yes, based on guide, it should be auto created, but I did not verify such document steps, if it is wrong, we can fix it | 07:31 |
cuihaozhi | Haomeng: ye, but my dnsmasq'interface("tapxxxxx") in br-int bridge, is that right? | 07:31 |
Haomeng | that should be devstack env for vms | 07:31 |
Haomeng | I think | 07:31 |
*** dlpartain has joined #openstack-ironic | 07:40 | |
*** subscope has joined #openstack-ironic | 07:40 | |
*** rameshg87 has joined #openstack-ironic | 08:05 | |
openstackgerrit | Ramakrishnan G proposed openstack/ironic-specs: Enable iscsi_ilo driver to share boot images https://review.openstack.org/137291 | 08:09 |
*** Masahiro has quit IRC | 08:11 | |
*** jistr has joined #openstack-ironic | 08:16 | |
*** Masahiro has joined #openstack-ironic | 08:16 | |
cuihaozhi | Haomeng: hi, i deleted error instance from nova,but ironic still display "the node reservation by $compute_node", what can i do for reuse the node? | 08:26 |
Haomeng | cuihaozhi: can you check from ironic side - ironic node-list | 08:26 |
Haomeng | cuihaozhi: to see if the instance_uuid is removed already | 08:27 |
cuihaozhi | Haomeng: instace_uuid is None, but node-show display reservation is $compute_hostname | 08:27 |
Haomeng | cuihaozhi: that should be wrong status | 08:27 |
Haomeng | cuihaozhi: the workaround is try to restart the conductor service and delete it by ironc node-delete | 08:28 |
cuihaozhi | Haomeng: api said "is locked by host localhost.localdomain", shold i modify database direct? | 08:30 |
Haomeng | cuihaozhi: :) | 08:30 |
Haomeng | cuihaozhi: go ahead:) | 08:31 |
cuihaozhi | Haomeng: :) | 08:32 |
Haomeng | cuihaozhi: :) | 08:33 |
Haomeng | cuihaozhi: suggest to delete and recreate another node to avoid more data issue | 08:33 |
cuihaozhi | Haomeng: ye.thanks for suggest :) | 08:34 |
Haomeng | :) | 08:35 |
*** athomas has joined #openstack-ironic | 08:38 | |
*** jcoufal has joined #openstack-ironic | 08:54 | |
*** ifarkas has joined #openstack-ironic | 08:59 | |
*** MattMan has joined #openstack-ironic | 09:00 | |
*** jiangfei has joined #openstack-ironic | 09:01 | |
rameshg87 | Haomeng, hi | 09:01 |
*** Masahiro has quit IRC | 09:05 | |
*** vdrok_ has joined #openstack-ironic | 09:07 | |
*** Masahiro has joined #openstack-ironic | 09:08 | |
*** vdrok has quit IRC | 09:08 | |
*** vdrok_ is now known as vdrok | 09:08 | |
vdrok | morning ironic | 09:08 |
*** naohirot has quit IRC | 09:10 | |
openstackgerrit | Ramakrishnan G proposed openstack/ironic-specs: New driver interface for RAID configuration https://review.openstack.org/135899 | 09:13 |
rameshg87 | morning vdrok | 09:14 |
vdrok | morning rameshg87 | 09:14 |
*** erwan_taf has joined #openstack-ironic | 09:14 | |
*** humble00 has joined #openstack-ironic | 09:15 | |
*** sambetts has joined #openstack-ironic | 09:16 | |
*** derekh has joined #openstack-ironic | 09:18 | |
*** humble00 has quit IRC | 09:25 | |
*** wanghua has joined #openstack-ironic | 09:26 | |
*** ndipanov has joined #openstack-ironic | 09:27 | |
cuihaozhi | Haomeng: follow the install-guide ,if i need add ipaddr to br-eth2? | 09:31 |
*** dtantsur|afk is now known as dtantsur | 09:34 | |
dtantsur | Morning! | 09:34 |
sambetts | Morning dtantsur! | 09:37 |
dtantsur | sambetts, o/ | 09:37 |
sambetts | dtantsur: I have a question about Vendor Passthru, whats the scope of functionality thats meant to be provided by them? | 09:38 |
dtantsur | sambetts, ideally, playing with vendor-specific functions that can't find their place in existing interfaces for now | 09:39 |
dtantsur | we're kind of misusing it though... | 09:39 |
sambetts | with the Cisco UCS driver, they want to use the vendor passthru to provide API access to faults and other UCS specfic information, however in comments on the Juno spec, there was a lot of adversity to making syncronous calls to UCSM to get this information | 09:42 |
sambetts | so I'm wondering what infrastructure is in place to allow a driver to provide that functionality asyncronously | 09:44 |
sambetts | or if that functionality even fits in the scope of Iroinc | 09:44 |
sambetts | s/Iroinc/Ironic | 09:45 |
*** romcheg has joined #openstack-ironic | 09:53 | |
*** igordcard has joined #openstack-ironic | 09:54 | |
*** Masahiro has quit IRC | 09:56 | |
cuihaozhi | hi, when i use ironic deploy phys-server, if i need to disable neutron-dhcp-agent? | 09:58 |
*** Masahiro has joined #openstack-ironic | 10:00 | |
*** kurtrao has quit IRC | 10:00 | |
*** kurtrao has joined #openstack-ironic | 10:00 | |
*** igordcard has quit IRC | 10:01 | |
*** Masahiro has quit IRC | 10:09 | |
sambetts | dtantsur: any ideas? or is there anywhere its documented? | 10:15 |
dtantsur | sambetts, well yeah, folks don't really like Ironic to make synchronous calls to the BMC. Why do you want to expose this kind of information? Is it used for deploy etc? | 10:16 |
sambetts | monitoring and exposing that information to the other services I believe | 10:18 |
dtantsur | sambetts, I'm not pretty sure it's within Ironic scope... | 10:20 |
*** igordcard has joined #openstack-ironic | 10:22 | |
sambetts | I think there is a misconception that Ironic == Baremetal for OpenStack instead of just provisioning | 10:22 |
sambetts | dtantsur: where do you think that baremetal monitoring etc fits within the Baremetal program? | 10:24 |
dtantsur | sambetts, I'm not sure, that's part of a long discussion what Ironic is and is not. Probably devananda is the best person to ask. Ceilometer is responsible for monitoring in OpenStack, generally speaking | 10:25 |
*** bradjones has joined #openstack-ironic | 10:25 | |
sambetts | dtantsur, does there not need to be a layer between ceilometer and the baremetel though? I've not seen much of Ceilometer, but I would have thought having drivers for baremetal would cross project boundries | 10:26 |
*** wanghua has quit IRC | 10:28 | |
dtantsur | sambetts, yeah, that's why I'm not 100% sure. This not the first time we're faced with this problem. Maybe worth raising it on the meeting? | 10:28 |
*** wanghua has joined #openstack-ironic | 10:29 | |
openstackgerrit | Dmitry Tantsur proposed openstack/ironic-specs: Introduce driver capabilities https://review.openstack.org/128927 | 10:29 |
sambetts | dtantsur, Yeah, I'll have to set my alarm for this one though, its quite early for our timezones isn't it? | 10:32 |
dtantsur | pretty much :( it's like 6am for me | 10:33 |
sambetts | 5am for me :( | 10:33 |
cuihaozhi | hello, if i need start neutron-dhcp-agent with boot instance from nova? | 10:36 |
*** pelix has joined #openstack-ironic | 10:36 | |
dtantsur | cuihaozhi, hi, IIRC yes | 10:40 |
*** sdake has quit IRC | 10:44 | |
*** andreykurilin_ has joined #openstack-ironic | 10:44 | |
openstackgerrit | Satoru Moriya proposed openstack/ironic: Use Literal Blocks to write code sample in docstring https://review.openstack.org/139965 | 10:44 |
rameshg87 | dtantsur, can you please have a look at https://review.openstack.org/#/c/139602/ | 10:46 |
smoriya | Hello, Ironic | 10:46 |
rameshg87 | dtantsur, needs an approval :) | 10:46 |
smoriya | I've just post a bug fix patch | 10:47 |
smoriya | This is my first patch to Ironic and so if I missed something please let me know :) | 10:47 |
rameshg87 | smoriya, hi, welcome .. :) | 10:49 |
smoriya | rameshg87, hi | 10:49 |
cuihaozhi | dtantsur: hi, how can i make "wait call back" instance failed? | 10:50 |
*** jistr has quit IRC | 10:51 | |
rameshg87 | cuihaozhi, you can just issue a delete request to the instance in "wait call back" | 10:52 |
cuihaozhi | rameshg87: thank u, but if i deleted the node, i should init the node infomations again, is there have another way? | 10:54 |
rameshg87 | cuihaozhi, i mean you can set the provisioning state to deleted | 10:54 |
rameshg87 | cuihaozhi, are you using nova with ironic ? | 10:54 |
cuihaozhi | rameshg87: yes i use nova. | 10:54 |
cuihaozhi | rameshg87:ok, nova delete :) | 10:55 |
rameshg87 | cuihaozhi, yes | 10:55 |
*** ramineni has quit IRC | 11:00 | |
*** rameshg87 has quit IRC | 11:03 | |
dtantsur | ram<TAB> | 11:03 |
dtantsur | oh c'mon folks, why can't you stay in channel? >_< | 11:03 |
pensu | Hi all, I am running Ironic devstack setup and I need to change DHCP config, but I don't see any dhcp.conf file anywhere..... | 11:04 |
pensu | the scenario is I don't want pxelinux.0 bootloader, I already have a bootloader, I directly want to transfer kernel and initrd images... | 11:05 |
sambetts | dtantsur: Do you use ZNC? | 11:06 |
*** jistr has joined #openstack-ironic | 11:09 | |
*** Masahiro has joined #openstack-ironic | 11:10 | |
pensu | but I can't figure out where does I change DHCP config.... | 11:10 |
*** smoriya has quit IRC | 11:12 | |
chenglch | pensu: dnsmasq | 11:14 |
*** Masahiro has quit IRC | 11:14 | |
chenglch | Hi all , I found ironic can set dhcp_provider=none, then how to use the external dhcp server? | 11:20 |
pensu | chenglch: dhsmasq doesn't use any config file either, the space is just empty "--config-file= " , that's it.....and the files it does use, don't seem to have any config options... | 11:21 |
chenglch | pensu: when you use nova boot, ironic will add data to --dhcp-optsfile | 11:24 |
dtantsur | sambetts, I do | 11:26 |
pensu | chenglch: okay, so there is no way I can go past bootloader? I mean it would be generated on fly, how exactly I stop it from passing pxelinux.0? | 11:29 |
cuihaozhi | Hi all, when i boot instance, the phys-node start with pxe report "ARP timeout";and tcpdump tapxxxx on ironic-conductor display not replay for dhcp request? where can i check it? | 11:30 |
openstackgerrit | Merged openstack/ironic: Fix for broken deploy of iscsi_ilo driver https://review.openstack.org/139602 | 11:31 |
chenglch | pensu: ironic.conf has tftp_root=' ', pxe_bootfile_name=' ' configuration. | 11:34 |
pensu | chenglch: yeah, that looks something nice.....will check it out....thanks...:) | 11:41 |
chenglch | pensu: :) | 11:42 |
*** lazy_prince is now known as killer_prince | 12:09 | |
*** luisjariz has joined #openstack-ironic | 12:09 | |
*** andreykurilin_ has quit IRC | 12:14 | |
*** jistr has quit IRC | 12:27 | |
*** jistr has joined #openstack-ironic | 12:28 | |
*** luisjariz has quit IRC | 12:32 | |
*** sambetts has quit IRC | 12:34 | |
openstackgerrit | Vladyslav Drok proposed openstack/ironic: Fix image_info passed to IPA for image download https://review.openstack.org/139988 | 12:41 |
*** pensu has quit IRC | 12:42 | |
*** sambetts has joined #openstack-ironic | 12:42 | |
*** Masahiro has joined #openstack-ironic | 12:44 | |
*** Masahiro has quit IRC | 12:49 | |
*** rameshg87 has joined #openstack-ironic | 13:01 | |
*** dprince has joined #openstack-ironic | 13:04 | |
*** athomas has quit IRC | 13:05 | |
*** alexpilotti has joined #openstack-ironic | 13:08 | |
*** athomas has joined #openstack-ironic | 13:14 | |
*** Haomeng has quit IRC | 13:21 | |
*** Haomeng has joined #openstack-ironic | 13:23 | |
*** naohirot has joined #openstack-ironic | 13:38 | |
*** chenglch has quit IRC | 13:50 | |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Store good MAC's and interfaces in node_info after processing https://review.openstack.org/140005 | 13:59 |
*** trown|call has joined #openstack-ironic | 14:03 | |
*** trown has quit IRC | 14:03 | |
*** trown|call has quit IRC | 14:04 | |
*** trown has joined #openstack-ironic | 14:05 | |
*** trown has quit IRC | 14:05 | |
*** trown has joined #openstack-ironic | 14:05 | |
*** krtaylor has quit IRC | 14:09 | |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Separate validating NIC data into a new plugin https://review.openstack.org/140010 | 14:13 |
openstackgerrit | Vladyslav Drok proposed openstack/ironic: Fix image_info passed to IPA for image download https://review.openstack.org/139988 | 14:13 |
dtantsur | ifarkas, could you review a couple of discoverd patches ^^^ please | 14:14 |
*** mikedillion has joined #openstack-ironic | 14:17 | |
ifarkas | dtantsur, sure thing ;-) | 14:21 |
*** ryanpetrello has joined #openstack-ironic | 14:22 | |
*** mjturek has joined #openstack-ironic | 14:23 | |
*** rloo has joined #openstack-ironic | 14:26 | |
*** krtaylor has joined #openstack-ironic | 14:30 | |
*** Masahiro has joined #openstack-ironic | 14:33 | |
*** Masahiro has quit IRC | 14:38 | |
*** jerryz has quit IRC | 14:40 | |
*** dlpartain has quit IRC | 14:44 | |
naohirot | jroll: good morning | 14:50 |
naohirot | NobodyCam: good morning | 14:51 |
naohirot | I just would like to let you know that I'm not able to attend Tuesday's IRC meeting, so I updated the white board iRMC status. | 14:52 |
naohirot | please find it | 14:53 |
naohirot | see you, good night ironic | 14:53 |
*** naohirot has quit IRC | 14:54 | |
NobodyCam | morning Ironic | 14:56 |
NobodyCam | morning naohirot | 14:57 |
*** rushiagr is now known as rushiagr_away | 15:06 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/ironic: Updated from global requirements https://review.openstack.org/139762 | 15:07 |
*** rameshg87 has quit IRC | 15:08 | |
*** rameshg87 has joined #openstack-ironic | 15:09 | |
dtantsur | morning, NobodyCam | 15:10 |
NobodyCam | morning dtantsur | 15:10 |
*** killer_prince is now known as lazy_prince | 15:13 | |
rloo | hello Ironickers and Pixie Boots | 15:19 |
dtantsur | rloo, hi :) | 15:20 |
sambetts | hello, rloo | 15:20 |
rloo | hey dtantsur, if you have a minute. I have a question about https://review.openstack.org/#/c/139602/1/ironic/common/images.py. (maybe I need more caffeine) | 15:21 |
dtantsur | sure | 15:21 |
rloo | dtantsur: why isn't the solution something like fetch(context, kernel_uuid, kernel_path, force_raw=CONF.force_raw_images)? | 15:21 |
rloo | afternoon sambetts! | 15:21 |
*** igordcard has quit IRC | 15:22 | |
dtantsur | lemme remember... | 15:23 |
NobodyCam | morning sambetts and rloo | 15:23 |
dtantsur | rloo, I'm not sure there's no point in talking about 'raw' kernel and ramdisk images, is it? | 15:23 |
rloo | dtantsur: then why does fetch() have that parameter? | 15:23 |
dtantsur | rloo, for instance images? | 15:24 |
* dtantsur is not sure | 15:24 | |
sambetts | Morning NobodyCam | 15:25 |
dtantsur | rloo, looks like image_cache does use force_raw. so IIRC the idea was that there's no point in raw images for kernel and ramdisk | 15:25 |
rloo | dtantsur: hmm. I looked at the patch that broke it but I didn't want to read the rationale behind that patch. There is something 'odd'. | 15:25 |
rloo | dtantsur: https://review.openstack.org/#/c/130880/ was the patch | 15:26 |
NobodyCam | :) | 15:27 |
dtantsur | rloo, I think it says that raw-vs-nonraw is not something that can be declared in the configuration; it should be detected by the driver | 15:27 |
*** dprince_ has joined #openstack-ironic | 15:36 | |
*** dprince has quit IRC | 15:36 | |
openstackgerrit | ZhiQiang Fan proposed openstack/ironic: rename oslo.concurrency to oslo_concurrency https://review.openstack.org/139782 | 15:38 |
openstackgerrit | Merged stackforge/ironic-discoverd: Store good MAC's and interfaces in node_info after processing https://review.openstack.org/140005 | 15:39 |
*** k4n0 has quit IRC | 15:40 | |
rloo | dtantsur: ok, that makes sense. thx. | 15:40 |
*** mikedillion has quit IRC | 15:40 | |
*** zz_jgrimm is now known as jgrimm | 15:45 | |
*** ndipanov has quit IRC | 15:50 | |
dtantsur | ifarkas, thank you for reviewing, please also have a look at https://review.openstack.org/#/c/140010 once you have some time | 15:53 |
openstackgerrit | Victor Lowther proposed openstack/ironic-specs: New Ironic provisioner state machine. https://review.openstack.org/133828 | 15:54 |
jroll | morning everybody :) | 15:54 |
rloo | hi dtantsur. wrt the driver-capabilities spec, I see you moved it to backlog, but didn't update it to reflect some questions I had. Should I copy/paste the same questions or not bother. It is possible I will be consistent and ask the same questions later :-) | 15:54 |
*** kbyrne has quit IRC | 15:55 | |
rloo | morning jroll | 15:55 |
NobodyCam | morning jroll | 15:55 |
*** rushiagr_away is now known as rushiagr | 15:55 | |
dtantsur | rloo, oh could you please at least leave a comment, mentioning that you had some questions? I'm afraid I don't have time for this spec though... | 15:55 |
*** ndipanov has joined #openstack-ironic | 15:56 | |
rloo | dtantsur: no worries. umm, do you want me to just update the backlog one then? cuz I think we want to land it still. | 15:57 |
NobodyCam | morning victor_lowther | 15:57 |
NobodyCam | :-p | 15:57 |
dtantsur | rloo, oh if you could update it, it would be awesome :) | 15:58 |
rloo | dtantsur: will do. | 15:58 |
dtantsur | thanks | 15:58 |
*** kbyrne has joined #openstack-ironic | 16:00 | |
ifarkas | dtantsur, yeah, I will do that too | 16:01 |
victor_lowther | morning | 16:02 |
dtantsur | victor_lowther, o/ | 16:06 |
*** david-lyle_afk is now known as david-lyle | 16:06 | |
openstackgerrit | Ruby Loo proposed openstack/ironic-specs: Introduce driver capabilities https://review.openstack.org/128927 | 16:10 |
*** rushiagr is now known as rushiagr_away | 16:12 | |
rameshg87 | ifarkas, hi | 16:16 |
*** sdake has joined #openstack-ironic | 16:17 | |
sdake | morning | 16:17 |
devananda | o/ | 16:18 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/ironic: Updated from global requirements https://review.openstack.org/139762 | 16:18 |
NobodyCam | morning rameshg87 sdake and devananda | 16:18 |
devananda | dtantsur, sambetts: hi! saw your ping. what's up? | 16:18 |
rameshg87 | NobodyCam, morning (though it's night here) :) | 16:18 |
NobodyCam | hehehehe :) | 16:18 |
dtantsur | devananda, morning. we were discussing with sambetts to what extend passthru should be used to provide vendor-specific information (e.g. health) from a node | 16:19 |
dtantsur | morning, sdake, rameshg87 | 16:19 |
sambetts | devananda, dtantsur: you beat me to it :-P | 16:19 |
ifarkas | rameshg87, hi | 16:20 |
rameshg87 | dtantsur, morning :) | 16:20 |
rameshg87 | ifarkas, got a chance to look at https://review.openstack.org/#/c/135899/ | 16:20 |
rameshg87 | ifarkas, ? | 16:21 |
devananda | sambetts: there is already a ceilometer<->ironic integration | 16:21 |
*** erwan_taf has quit IRC | 16:21 | |
*** Marga_ has joined #openstack-ironic | 16:21 | |
*** Marga_ has quit IRC | 16:21 | |
sambetts | devananda, dtantsur: I've been asked to look over the Cisco UCSM plugin work, and I was going through your comments on the specs posted for the Juno cycle, and there was a lot of discussion around the use of syncronous vendor calls to get things like the nodes firmware version etc. | 16:21 |
rameshg87 | ifarkas, i think it's time we got somebody to look at that | 16:21 |
ifarkas | rameshg87, yeah, I have already seen the update. What are the changes? | 16:22 |
sdake | hey guys, my deploy ramdisk locks up when trying to boot fedora atomic, can anyone take a look and offer suggestions on next debug steps? | 16:22 |
sdake | http://paste.fedoraproject.org/157504/18016744 | 16:22 |
*** Masahiro has joined #openstack-ironic | 16:22 | |
rameshg87 | ifarkas, i just removed the commit msg to say "it's ready" | 16:22 |
*** Marga_ has joined #openstack-ironic | 16:22 | |
*** achanda has joined #openstack-ironic | 16:22 | |
*** mikedillion has joined #openstack-ironic | 16:22 | |
rameshg87 | ifarkas, only the commit msg was changed | 16:22 |
rameshg87 | ifarkas, we need someone to take a look at it :) | 16:23 |
devananda | sambetts: with some proposals to revamp that, as it wasn't handling vendor specifics outside of IPMI very well | 16:23 |
rameshg87 | sdake, guess it has notified ironic conductor to continue the installation, isn't ironic-conductor doing something with that bare metal ? | 16:23 |
jroll | sdake: I don't know what this means, but seems like something that should be looked into maybe Dec 8 05:28:24 (none) daemon.err tgtd: iser_ib_init(3351) Failed to initialize RDMA; load kernel modules? | 16:24 |
sambetts | devananda: Ah, I was just writing to ask about that, because the UCSM driver won't be using IPMI | 16:24 |
devananda | sambetts: yea. synchronous calls are pretty heavily discouraged. there have been several discussious as to why ... | 16:24 |
*** anderbubble has joined #openstack-ironic | 16:25 | |
rameshg87 | ifarkas, are you planning to pursue this still: https://review.openstack.org/107981 ? | 16:25 |
ifarkas | rameshg87, yes, once the general raid interface spec is accepted | 16:25 |
sambetts | devananda: Yes, I heard many of them at the Summit | 16:25 |
devananda | sambetts: have a look at these specs: https://review.openstack.org/#/q/owner:%22Jim+Mankovich%22+status:open,n,z | 16:26 |
devananda | sambetts: those cover the proposed changes to ironic and ceilometer to make it easier to support | 16:26 |
devananda | sambetts: non-IPMI-based extensions to monitoring | 16:26 |
rameshg87 | ifarkas, i guess you will revise the spec to implement the interfaces that we defined in 135899, right ? | 16:26 |
*** Masahiro has quit IRC | 16:26 | |
ifarkas | rameshg87, yeah, exactly | 16:26 |
rameshg87 | ifarkas, okay | 16:27 |
devananda | sambetts: I'm sure jmanko would apprecieate some more eyes / input on them from other stakeholders, too | 16:27 |
rameshg87 | devananda, can you please have a look at https://review.openstack.org/#/c/137291/ , you had questions and i had replied as comments inline. | 16:27 |
sambetts | devananda: thats great thanks! s | 16:28 |
sambetts | devananda: My only other question would be about information such as something like firmware version which isn't going to change regularly and doesn't really fit into the sensor monitoring model, you mentioned in the juno spec about storing it in the ironic DB, so that when queried it simply retreived it from there? what would be the process to keep that information in the DB up to date? | 16:31 |
devananda | sambetts: we don't have a great answer for that in the current code, but its being worked into this cycle's theme - https://review.openstack.org/133828 | 16:32 |
*** marcoemorais has joined #openstack-ironic | 16:35 | |
*** achanda has quit IRC | 16:37 | |
sambetts | devananda: Great! I'll keep my eye on those 2 blueprints, thanks :D | 16:37 |
*** achanda has joined #openstack-ironic | 16:38 | |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Separate validating NIC data into a new plugin https://review.openstack.org/140010 | 16:38 |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Wait for power off state before calling discovery done https://review.openstack.org/140064 | 16:39 |
devananda | sambetts: well, Jim has four BP up :) | 16:39 |
sdake | jroll rdma is infiniband - I am pretty sure we don't need infiniband drivers | 16:39 |
* sambetts looks | 16:39 | |
sdake | rameshg87 at that point in the console log, the deploy locks up | 16:40 |
sdake | I just sorted out there is a magic port 10000 | 16:40 |
jroll | sdake: dunno, that error comes from tgtd which is the iscsi daemon I believe | 16:40 |
*** marcoemorais1 has joined #openstack-ironic | 16:40 | |
sdake | jroll same rror comes out via cirros deploy | 16:41 |
sdake | cirros deploy works | 16:41 |
jroll | huh, idk | 16:41 |
rameshg87 | sdake, "nullwaiting for notice of complete" says it has notified ironic-conductor to continue the deployment | 16:41 |
rameshg87 | sdake, what do you see in ironic-conductor logs ? | 16:41 |
sdake | cool thats a good bone | 16:41 |
*** marcoemorais has quit IRC | 16:42 | |
sdake | rameshg87 my logs are long gone from last night, but I'll get it going again in a bit | 16:42 |
sdake | I notice there isa magic port 10000 that should be triggered by ironic | 16:42 |
rameshg87 | sdake, yeah, | 16:42 |
*** achanda has quit IRC | 16:42 | |
sdake | i'm going to put some debug in and see if that isn't happening and why | 16:42 |
rameshg87 | sdake, that is to indicate the bare metal that i am done with deployment and restart | 16:42 |
sdake | you mentioned nullwaiting | 16:42 |
sdake | where is that? | 16:42 |
sambetts | devananda: thanks v.much :-) | 16:43 |
jroll | sdake: I'd wonder if 1000 is open in your firewall | 16:43 |
rameshg87 | sdake, you wanted to debug on the bare metal or you wanted to debug on ironic-conductor ? | 16:43 |
*** marcoemorais1 has quit IRC | 16:43 | |
sdake | I run without a firewall typically ;-) | 16:43 |
sdake | well I am running in devstack with pxe+ssh driver | 16:43 |
jroll | ok, just making sure | 16:44 |
*** jgrimm has quit IRC | 16:44 | |
sdake | the deploy ramdisk is the same between cirros or atomic | 16:44 |
*** marcoemorais has joined #openstack-ironic | 16:45 | |
rameshg87 | sdake, i would suggest run the same thing again and watch the ironic-conductor logs | 16:46 |
rameshg87 | sdake, because the deploy ramdisk seems to have done it's part :) | 16:46 |
*** achanda has joined #openstack-ironic | 16:46 | |
sdake | the null wait, that indicates the ramdisk has been deployed? | 16:46 |
sdake | rather the image the ramdisk wants ? | 16:46 |
sdake | its getting hung up on the nc I suspect | 16:46 |
*** marcoemorais has quit IRC | 16:47 | |
rameshg87 | sdake, yes, deploy ramdisk is waiting here actually: https://github.com/openstack/diskimage-builder/blob/master/elements/deploy-ironic/init.d/80-deploy-ironic#L56-L57 | 16:48 |
*** pensu has joined #openstack-ironic | 16:48 | |
sdake | yup | 16:48 |
*** romcheg has quit IRC | 16:49 | |
*** romcheg has joined #openstack-ironic | 16:49 | |
rameshg87 | sdake, and once ironic-conductor completes deployments it is supposed to do this: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/deploy_utils.py#L391-L398 | 16:49 |
rameshg87 | sdake, so you are getting stuck somewhere before actually | 16:49 |
*** romcheg has quit IRC | 16:50 | |
rameshg87 | NobodyCam, hi | 16:50 |
rameshg87 | NobodyCam, what's pet / snowflake care ? https://review.openstack.org/#/c/135899/6/specs/backlog/raid-interface.rst | 16:50 |
sdake | rameshg87 oddly looking at the logs, there is no run of the echo near that netcat | 16:51 |
clif_h | HackThePlanet | 16:51 |
*** Marga_ has quit IRC | 16:51 | |
clif_h | lol >_< | 16:51 |
rameshg87 | sdake, i didn't get you | 16:51 |
sdake | oh "waiting for notice" | 16:51 |
sdake | it says "nullwaiting for notice" | 16:51 |
rameshg87 | sdake, ah | 16:51 |
rameshg87 | sdake, the "null" is actually from curl | 16:52 |
sdake | wonder what the null print is | 16:52 |
rameshg87 | sdake, the command before | 16:52 |
sdake | right | 16:52 |
*** sambetts_ has joined #openstack-ironic | 16:52 | |
NobodyCam | rameshg87: treatning nodes as unique snowflakes, | 16:52 |
*** sambetts has quit IRC | 16:53 | |
NobodyCam | rameshg87: we have used the terms pet vs cattle | 16:55 |
rameshg87 | NobodyCam, still it's not clear to me :) | 16:55 |
rameshg87 | NobodyCam, may be plain english is required to educate me | 16:55 |
NobodyCam | ironic should treat modes as cattle, all the same, nothing unique | 16:56 |
rameshg87 | NobodyCam, and what is "modes" here ? | 16:56 |
NobodyCam | rameshg87: nodes :_p | 16:57 |
devananda | http://www.computerweekly.com/blogs/cwdn/2013/04/treat-cloud-servers-like-cattle-not-puppies.html | 16:57 |
devananda | going even further - http://robhirschfeld.com/2014/03/17/mayflies/ | 16:57 |
* NobodyCam has only had two cups of coffee | 16:57 | |
*** rushiagr_away is now known as rushiagr | 16:57 | |
NobodyCam | ahh thank you devananda | 16:57 |
* rameshg87 wonders what this is all about | 16:57 | |
devananda | rameshg87: the first link above may be more helpful for background | 16:58 |
rameshg87 | devananda, thanks :) | 16:58 |
*** sambetts_ is now known as sambetts | 16:58 | |
devananda | hum... it's more marketing speak than I had expected from the title :( | 16:59 |
NobodyCam | brb | 17:00 |
devananda | rameshg87: better link: http://www.theregister.co.uk/2013/03/18/servers_pets_or_cattle_cern/ | 17:00 |
devananda | with cute pictures, lol | 17:00 |
*** romcheg has joined #openstack-ironic | 17:02 | |
*** krtaylor has quit IRC | 17:04 | |
*** krtaylor has joined #openstack-ironic | 17:06 | |
openstackgerrit | Merged stackforge/ironic-discoverd: Separate validating NIC data into a new plugin https://review.openstack.org/140010 | 17:07 |
*** PaulCzar has joined #openstack-ironic | 17:09 | |
*** anderbubble has left #openstack-ironic | 17:11 | |
rameshg87 | victor_lowther, now i got one more which i am googling for the meaning https://review.openstack.org/#/c/135899/6 | 17:12 |
victor_lowther | rameshg87: hmmm? | 17:13 |
rameshg87 | victor_lowther, i hope you meant to generalize raid config across all driver | 17:13 |
rameshg87 | victor_lowther, am i correct ? :) | 17:13 |
victor_lowther | Yes. | 17:13 |
* dtantsur also finally understood this pet-cattle thing | 17:14 | |
NobodyCam | :) | 17:14 |
*** anderbubble_ has joined #openstack-ironic | 17:14 | |
*** anderbubble_ has joined #openstack-ironic | 17:15 | |
victor_lowther | Specifying target size, RAID level, number of disks, whether a volume can share physical disks with another volume, interface type, and media type should be enough info for a lower-level driver to intelligently lay out RAID volumes. | 17:15 |
*** anderbubble_ is now known as anderbubble | 17:16 | |
rameshg87 | victor_lowther, why would it need number of disks ? | 17:16 |
rameshg87 | victor_lowther, that is dependent on the physical disks that the machine is having, right ? | 17:16 |
victor_lowther | and as long as you have reasonable fallback rules about what to do with interface type and media type you can potentially make those optional. | 17:16 |
victor_lowther | because saying "make me a RAID 5" can bedone with just 3 disks | 17:17 |
*** ndipanov has quit IRC | 17:17 | |
victor_lowther | or all 20 something of the,. | 17:17 |
victor_lowther | Pretty much only RAID1 has a good default on number of disks to use. | 17:17 |
*** athomas has quit IRC | 17:18 | |
victor_lowther | and even that can be hazy if your RAID controller understands RAID1e. | 17:18 |
*** openstackgerrit has quit IRC | 17:19 | |
*** openstackgerrit has joined #openstack-ironic | 17:20 | |
*** tylerdurden has quit IRC | 17:20 | |
victor_lowther | more to the point, I do not expect folks riding herd on Ironic to live in total ignorance of how any physical disks a machine has. | 17:20 |
*** rameshg87 has quit IRC | 17:20 | |
victor_lowther | because you have to plan from something. | 17:20 |
*** rameshg87 has joined #openstack-ironic | 17:20 | |
rameshg87 | victor_lowther, okay | 17:22 |
victor_lowther | and (aside from people using RAID5 on modern hardware being functionally insane), the two mode common RAID layouts I use on cloudy type workloads are: | 17:22 |
victor_lowther | 1 big RAID10 | 17:22 |
victor_lowther | 1 RAID 1 for the OS, everything else as a JBOD | 17:22 |
victor_lowther | first for non-storage nodes | 17:22 |
devananda | victor_lowther: at some level, there *has* to be a systematic awareness of how many disks that machine should have. If nothing else, for fault detection. | 17:22 |
victor_lowther | second for stirage nodes. | 17:23 |
victor_lowther | devananda: yes | 17:23 |
devananda | victor_lowther: yup. those are the same two I've seen most commonly. | 17:23 |
devananda | though there are people who still use RAID6 eg. for storage-appliance nodes | 17:24 |
victor_lowther | yeah | 17:27 |
victor_lowther | but even then, you need to tell the RAID driver how many disks to use for that RAID6 | 17:27 |
victor_lowther | although allowing 'min' and 'max' might be decent values to allow as well. | 17:28 |
victor_lowther | no matter how much my inner type purist loathes the idea. :) | 17:28 |
devananda | victor_lowther: what if we express it as: no_raid, mirrored_root, all_raid(level=N) | 17:29 |
victor_lowther | eh. | 17:29 |
devananda | rather than attempt to model all arbitrary possibilities, which is not an exercise I look forward to | 17:30 |
*** ndipanov has joined #openstack-ironic | 17:30 | |
victor_lowther | hm? | 17:31 |
rameshg87 | devananda, by all_raid, do you mean to put all physical disks into raid level N ? | 17:31 |
devananda | in the fullness of time, the disk/raid topology needs to be a flavor extra_spec, so Nova can schedule on it | 17:31 |
victor_lowther | I have no intention of modelling all arbitrary posibilities. | 17:31 |
devananda | rameshg87: yup | 17:32 |
victor_lowther | but if we want to deliberately restrict what can be done in a driver-independent fashion I am OK with that. | 17:32 |
devananda | victor_lowther: that's what I'm thinking, yes. | 17:33 |
victor_lowther | the crazy people with their weird requirements can use vendor_passthrough. | 17:33 |
devananda | victor_lowther: rather than allowing something either free-form (which ends up being driver-specific) or piggybacking on cinder volume specs (which has been floated several times, but not gotten any real traction over the last four cycles) | 17:33 |
*** lazy_prince is now known as killer_prince | 17:34 | |
*** Marga_ has joined #openstack-ironic | 17:36 | |
rameshg87 | devananda, and what's mirrored_root ? RAID 1 ? | 17:37 |
*** Marga_ has quit IRC | 17:37 | |
devananda | rameshg87: yes. this covers the options that victor_lowther already described | 17:37 |
jroll | I find it interesting how much RAID complicates ironic | 17:37 |
devananda | jroll: frustrating, too | 17:38 |
victor_lowther | jroll: it is a large body of legacy knowledge | 17:38 |
jroll | devananda: right, I was trying to put it lightly | 17:38 |
victor_lowther | that vms are lergely insulated from | 17:38 |
jroll | :) | 17:38 |
devananda | thinking about it breaks the abstraction | 17:38 |
*** Marga_ has joined #openstack-ironic | 17:38 | |
devananda | at least, thinking about it in the way we are accustomed to :) | 17:38 |
victor_lowther | it breaks Nova's abstractions, sure. | 17:39 |
devananda | right - it breaks the compute-instance abstraction | 17:39 |
NobodyCam | wow lots of lag | 17:39 |
victor_lowther | but it is a thing you have to care about for redundancy, uptime, and performance reasons on metal. | 17:39 |
jroll | right, I understand why we need to consider it, but it just sucks :| | 17:40 |
devananda | victor_lowther: redundancy and uptime are not things I should care about w.r.t. individual instances in a cloud model | 17:40 |
victor_lowther | because even though physical boxes are cattle, killing them out of schedule is Bad and you still force-feed them antibiotics. :) | 17:41 |
devananda | heh | 17:41 |
jroll | haha | 17:41 |
jroll | I'm a huge fan of using the cloud model for instances built by ironic | 17:42 |
jroll | however I don't think everyone is | 17:42 |
sdake | rameshg87 does the system that is being deployed require alot of memory to deal with the iscsi stuff (for a 6gb image) | 17:42 |
devananda | so, what do ya'll think of, as an initial sketch of an LCD for modelling RAIDs within Ironic .... | 17:42 |
jroll | not sure if that's good or bad | 17:42 |
devananda | * should there be a RAID? yes/no | 17:42 |
devananda | * should it be a RAID1 of the first two disks? yes/no | 17:42 |
devananda | * should it be a RAID(N) of all the disks? yes/no | 17:43 |
victor_lowther | for suitable definition of "first" | 17:43 |
devananda | right -- the details are up to each driver | 17:43 |
victor_lowther | devananda: Not too sure of that. | 17:43 |
devananda | k k. it's just a strawman | 17:44 |
*** pcrews has joined #openstack-ironic | 17:44 | |
victor_lowther | For a concrete example, take a Dell R720XD. | 17:44 |
victor_lowther | Front of machine is all spindles | 17:44 |
victor_lowther | either 12 3.5" disks or 24 2.5" | 17:44 |
PaulCzar | is there a way I can set extra dhcp options via ironic? or do I need to run neutron port-update to do it ? | 17:44 |
victor_lowther | It is a common ask to put the OS on the two 2.5" spindles in the rear | 17:45 |
devananda | I actually dont want to spend much time on it -- going to take a few things off my backlog, then try to to the state model I started last week | 17:45 |
devananda | victor_lowther: sure - and that would reasonably be a driver optimization, IMO | 17:45 |
devananda | victor_lowther: my point is, the location of the OS RAID is not something which Nova needs to schedule on | 17:46 |
victor_lowther | and where those appear from the raid controller's POV depends on the controller, the backplanes involved, and how everything is cabled internally. | 17:46 |
victor_lowther | Right, it is not. | 17:46 |
devananda | victor_lowther: whether the node *has* a two-disk RAID for the OS is something Nova cares about | 17:46 |
victor_lowther | Nova should not know or care. | 17:46 |
victor_lowther | but Ironic should | 17:46 |
devananda | well. the driver should. | 17:46 |
devananda | I don't see why ConductorManager should care | 17:46 |
victor_lowther | More to the point, the operator should. | 17:47 |
devananda | fair | 17:47 |
*** Marga_ has quit IRC | 17:47 | |
victor_lowther | and they should be able to give the driver enough info to place the virtual drives in approximately the right location when the RAID is being ZAPPED into existence. | 17:47 |
devananda | perhaps a way for operator to inform the driver of preferences, driver to execute those preferences (or error) on the hardware, | 17:47 |
victor_lowther | After that, it should be hands off. | 17:48 |
devananda | so we need two interfaces defined (well, more than that, but the rest will fall out from these) | 17:48 |
victor_lowther | and that is what I thought the JSON blob in rameshg87's spec was talking about. | 17:48 |
devananda | nova-ironic, for describing the node properties which inform scheduling | 17:48 |
devananda | operator-driver, for suggesting particulars about how the raid on *that* machine should be set up | 17:49 |
devananda | because, yes, the operator does, at some point, need to care about each machine | 17:50 |
victor_lowther | that class of machine, at least | 17:50 |
victor_lowther | to keep them from becoming pets. :) | 17:50 |
devananda | even if that's a script which asserts a given behavior across an identical fleet | 17:50 |
jroll | ++ for class | 17:50 |
devananda | right | 17:50 |
victor_lowther | that is wondering more into orchestration territory. | 17:51 |
devananda | jroll: I'm loathe to encode orchestration into Ironic's API, which "assert this change across this class of machines" would require | 17:51 |
devananda | heh | 17:51 |
jroll | ha | 17:51 |
jroll | devananda: it should be on the operator side | 17:51 |
devananda | jroll: ++ | 17:51 |
devananda | I'm all for providing tools to enable that | 17:52 |
devananda | eg, tagging | 17:52 |
victor_lowther | the operator-ironic API should expose some suitably abstracted physical disk info | 17:52 |
jroll | for node in the_relevant_nodes_i_found: ironic node-update replace properties/raidstuff=blah | 17:52 |
jroll | yeah | 17:52 |
devananda | jroll: yes, exactly | 17:52 |
victor_lowther | and allow you to push a semi-detailed raid config to it | 17:53 |
victor_lowther | once the arrays are created the nova-ironic API can update the relavent node properties for scheduling purposes. | 17:53 |
victor_lowther | and enforcing the same RAID layout across a given class of machines then becomes the orchestrator's problem | 17:55 |
victor_lowther | be it heat, crowbar, or whatever. | 17:55 |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: split discoverd.py into 2 modules https://review.openstack.org/140102 | 17:56 |
rameshg87 | victor_lowther, but are the *semi-details* only the LCDs that we talked about ? | 17:56 |
victor_lowther | From my POV, more the comments I made to your spec | 17:57 |
victor_lowther | push something detailed enough for the RAID driver to figure it out for itself | 17:58 |
victor_lowther | or push an exact config (although that will probably be vendor-specific) | 17:58 |
NobodyCam | brb | 17:58 |
*** Marga_ has joined #openstack-ironic | 17:59 | |
*** Marga_ has quit IRC | 17:59 | |
rameshg87 | ifarkas, hi | 18:00 |
rameshg87 | ifarkas, i am not sure if you have been following the above ^^ | 18:00 |
*** achanda has quit IRC | 18:01 | |
*** derekh has quit IRC | 18:02 | |
*** achanda has joined #openstack-ironic | 18:02 | |
rameshg87 | victor_lowther, devananda, thanks i will revise the spec | 18:02 |
openstackgerrit | Merged openstack/ironic: Updated from global requirements https://review.openstack.org/139762 | 18:03 |
*** dlaube has joined #openstack-ironic | 18:04 | |
devananda | rameshg87: I just posted a comment | 18:05 |
rameshg87 | devananda, thanks, let me take a look | 18:05 |
*** achanda has quit IRC | 18:06 | |
vdrok | JayF, jroll, hi! I have a question about ipa | 18:07 |
jroll | vdrok: don't ask to ask, just ask :) | 18:07 |
vdrok | and hi everyone | 18:07 |
jroll | also, hi :D | 18:08 |
vdrok | jroll, ipa requires to have checksum in image_info that's passed to it | 18:08 |
vdrok | to download image | 18:08 |
jroll | vdrok: yes | 18:08 |
vdrok | jroll, but if the image is not from glance, what is the preferred way to handle it? | 18:09 |
vdrok | require to include it in instance_info? | 18:09 |
vdrok | or make it optional on ipa side? | 18:09 |
rameshg87 | sdake, it doesn't need a lot of memory. iscsi stuff exposes local disk and ironic-conductor writes the image, so most of the work is take care by ironic-conductor | 18:09 |
jroll | vdrok: how does this work without glance? do you have code? | 18:10 |
JayF | I don't think it's a good idea to download and image a disk without checking the hash of the image | 18:10 |
jroll | vdrok: ^ | 18:10 |
JayF | regardless of the source of the image. | 18:10 |
jroll | yeah | 18:10 |
*** Marga_ has joined #openstack-ironic | 18:10 | |
*** Masahiro has joined #openstack-ironic | 18:11 | |
vdrok | jroll, jayf, there's no code for ipa driver now, there's something here, but its wip https://review.openstack.org/#/c/136741/ | 18:11 |
*** jistr has quit IRC | 18:11 | |
vdrok | maybe just pass the url in image_info urls | 18:11 |
jroll | vdrok: ah, right, I see | 18:11 |
*** Marga_ has quit IRC | 18:12 | |
rameshg87 | jroll, regarding https://bugs.launchpad.net/ironic/+bug/1394609 | 18:12 |
jroll | rameshg87: yes? | 18:12 |
rameshg87 | jroll, instead of putting bash scripts inside IPA ramdisk | 18:12 |
rameshg87 | jroll, why not enhance IPA to add a command to expose a disk over iscsi | 18:13 |
vdrok | jroll, JayF, then will be adding checksum to instance_info, thanks | 18:13 |
rameshg87 | jroll, and then we should be able to use it with existing infrastructure | 18:13 |
jroll | rameshg87: I want this to work with the iscsi deploy driver without changing the driver :) | 18:13 |
*** Marga_ has joined #openstack-ironic | 18:13 | |
jroll | rameshg87: so I don't want ironic to have to call a command | 18:13 |
rameshg87 | jroll, hmm, okay | 18:14 |
*** Masahiro has quit IRC | 18:15 | |
*** Marga_ has quit IRC | 18:16 | |
rameshg87 | jroll, yeah makes sense. but i still wonder why to bring bash scripts in ? | 18:17 |
rameshg87 | jroll, why not add functionality to IPA that acts based on arguments :) | 18:18 |
jroll | rameshg87: maybe, I'd be ok with that :) | 18:18 |
rameshg87 | jroll, like whatever you had mentioned deploy_method="iscsi" or deploy_method="agent" (default) | 18:19 |
JayF | I'd even make it more like | 18:19 |
JayF | emulate-pxe-ramdisk on the kernel commandline | 18:19 |
JayF | and if you don't get it | 18:19 |
*** mikedillion has quit IRC | 18:19 | |
jroll | right, yeah | 18:19 |
JayF | erm, if you get the setting, you'd not start IPA, and instead have the iscsi stuff start | 18:20 |
*** harlowja_away is now known as harlowja | 18:20 | |
jroll | I was just thinking that copying a bash script would be easier than porting it to python :P | 18:20 |
*** mikedillion has joined #openstack-ironic | 18:20 | |
sdake | rameshg87 cool thanks for the info, I'll keep debugging, iscsi_deploy.continue_deploy did the job, but it may not be returning a root_uuid | 18:20 |
sdake | the 6gig dd takes a bit of time | 18:20 |
rameshg87 | sdake, won't take that long, keep looking :) | 18:21 |
sdake | on the plus side, I've become an expert in how ironci works ;-) | 18:21 |
rameshg87 | sdake, great :) | 18:21 |
sdake | I really like the architecture | 18:22 |
sdake | versioned objects rock | 18:22 |
sdake | I've copied alot of ironic into magnum | 18:22 |
*** mikedillion has quit IRC | 18:22 | |
sdake | probably would have been faster to fork ironic into magnum but then I wouldn't have understood the codebase | 18:22 |
openstackgerrit | Merged openstack/python-ironicclient: VendorPassthru commands to support different HTTP methods https://review.openstack.org/134253 | 18:23 |
rameshg87 | jroll, okay, but it's not a lot of code in there in PXE ramdisk. bringing into one code base seems better to me. :) | 18:24 |
*** Marga_ has joined #openstack-ironic | 18:24 | |
sdake | has anyone thought of making the pxe+ssh use pxe+libvirt for power control instead? | 18:24 |
jroll | rameshg87: that's fine, I would +1 that patch :) | 18:25 |
sdake | atm, if a instance gets wedged, ironic gets wedged | 18:25 |
jroll | sdake: the ssh driver basically ssh's somewhere (sometimes localhost) and runs virsh, VBoxControl, etc depending on the hypervisor | 18:25 |
sdake | i see | 18:26 |
sdake | thanks that helps | 18:26 |
jroll | np | 18:26 |
jroll | sdake: a cool use case for that is local dev | 18:26 |
* rameshg87 realises it's time to go to bed | 18:26 | |
jroll | run 3 VMs on your laptop, one runs devstack, the other 2 are for ironic to test with | 18:26 |
sdake | ya agree thatis how I am using it | 18:26 |
jroll | ironic can ssh back | 18:26 |
*** Marga_ has quit IRC | 18:26 | |
*** rameshg87 has quit IRC | 18:26 | |
sdake | but nova delete doesn't actually delete | 18:27 |
sdake | it gets stuck | 18:27 |
jroll | that sounds like a bug | 18:28 |
jroll | or something weird in configs, that should work | 18:28 |
sdake | agree | 18:28 |
NobodyCam | have a good night ramesh87 | 18:28 |
NobodyCam | ramshg87 even | 18:29 |
*** igordcard has joined #openstack-ironic | 18:29 | |
jroll | sdake: I should say that does work, all the time, in devstack in the gate | 18:29 |
sdake | reproducer is to use this image: http://dl.fedoraproject.org/pub/alt/stage/21_RC5/Cloud/Images/x86_64/Fedora-Cloud-Atomic-20141203-21.x86_64.qcow2 | 18:29 |
*** andreykurilin_ has joined #openstack-ironic | 18:33 | |
*** pelix has quit IRC | 18:39 | |
*** marcoemorais has joined #openstack-ironic | 18:42 | |
*** ndipanov is now known as ndipanov_gone | 18:47 | |
*** sdake has quit IRC | 18:47 | |
*** Marga_ has joined #openstack-ironic | 18:48 | |
*** Marga_ has quit IRC | 18:48 | |
*** eghobo has joined #openstack-ironic | 18:49 | |
*** eghobo has quit IRC | 18:52 | |
*** igordcard has quit IRC | 18:55 | |
*** Marga_ has joined #openstack-ironic | 18:57 | |
*** marcoemorais has quit IRC | 18:59 | |
*** marcoemorais has joined #openstack-ironic | 18:59 | |
*** marcoemorais has quit IRC | 18:59 | |
*** marcoemorais has joined #openstack-ironic | 18:59 | |
*** davideagnello has joined #openstack-ironic | 19:00 | |
*** jiangfei has quit IRC | 19:01 | |
*** jiangfei has joined #openstack-ironic | 19:02 | |
*** spandhe has joined #openstack-ironic | 19:02 | |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: split discoverd.py into 2 modules https://review.openstack.org/140102 | 19:06 |
dtantsur | ifarkas is gonna hate me for this patch ^^^ :D | 19:06 |
dtantsur | but that will be tomorrow :) and now calling it a day, g'night everyone | 19:07 |
*** dtantsur is now known as dtantsur|afk | 19:07 | |
*** eghobo has joined #openstack-ironic | 19:08 | |
*** Marga_ has quit IRC | 19:09 | |
*** pensu has quit IRC | 19:09 | |
*** igordcard has joined #openstack-ironic | 19:10 | |
*** dprince_ has quit IRC | 19:11 | |
*** Marga_ has joined #openstack-ironic | 19:12 | |
*** marcoemorais has quit IRC | 19:12 | |
*** Marga_ has quit IRC | 19:13 | |
*** marcoemorais has joined #openstack-ironic | 19:13 | |
*** Marga_ has joined #openstack-ironic | 19:13 | |
*** Marga_ has quit IRC | 19:17 | |
NobodyCam | noght dtantsur|afk | 19:18 |
NobodyCam | night even | 19:18 |
NobodyCam | general question, as an operator are tenants of bare metal nodes allowed to reflash bios/device roms? | 19:20 |
NobodyCam | question is comming from https://review.openstack.org/#/c/133902 | 19:21 |
Shrews | NobodyCam: that would be an odd thing to allow IMO, but i'm not an operator | 19:22 |
victor_lowther | I would put that in the "hahaha... oh, you are serious. HAHAHAHA!" category. | 19:23 |
Shrews | oh, i watched that talk a while back | 19:23 |
NobodyCam | I was thinking of some type of "hot fix" that the tenant needed / wanted. | 19:23 |
*** eghobo has quit IRC | 19:23 | |
Shrews | NobodyCam: the talk made sense at the time, but i don't remember the "tenant reflashing bios" part | 19:24 |
openstackgerrit | Ruby Loo proposed openstack/ironic: Update dev quick-start for devstack https://review.openstack.org/140124 | 19:24 |
victor_lowther | yeah, I was the guy who kept asking how they can prevent BIOS flashes to start with as opposed to just catching it after the fact. | 19:24 |
*** erwan_taf has joined #openstack-ironic | 19:24 | |
victor_lowther | well, that is the thing. | 19:25 |
victor_lowther | once someone has root, most systems cannot stop the tenants from flashing the bios and firmware | 19:25 |
NobodyCam | with the spec I linked above that would cause the node to fail the delete / taredown tests | 19:25 |
victor_lowther | the most they can do is catch it after the fact and reflash | 19:26 |
NobodyCam | ya and some times there is a valid reason to flash a firmware | 19:26 |
victor_lowther | right, there are. | 19:26 |
victor_lowther | but the tenant should never be doing it by themselves. | 19:26 |
victor_lowther | because they are leasing the hardware, they do not own it. | 19:27 |
*** dprince has joined #openstack-ironic | 19:27 | |
victor_lowther | and it opens too many attack vectors. | 19:27 |
NobodyCam | so request a node with the new requirments and migrate the data ... tare down the old nod | 19:27 |
victor_lowther | yes, and hope they did not backdoor the firmware. | 19:28 |
* NobodyCam assumes the new firmware would come from a trusted source | 19:28 | |
victor_lowther | TXT lets you see if the firmware has been tampered | 19:28 |
devananda | in all such discussions, we need to distinguish between untrusted multi-tenant, trusted multi-tenant, and single-tenant | 19:29 |
devananda | we've got users in each of those categories today | 19:29 |
victor_lowther | NobodyCam: and if I were an attacker with sufficent resources, I would let you think that it had. | 19:29 |
devananda | unfortunately, TXT does not address SATA or USB firmware .... | 19:30 |
victor_lowther | Indeed | 19:30 |
victor_lowther | and most firmware/hardware does not have a mechanism to prevent flashes in the first place. | 19:30 |
victor_lowther | which sucks. | 19:31 |
victor_lowther | so instead we just get to operate with the risk that someone has pwnz0red the system, and hopefully we can detect that. | 19:32 |
*** Marga_ has joined #openstack-ironic | 19:32 | |
devananda | yup | 19:32 |
*** zz_jgrimm has joined #openstack-ironic | 19:33 | |
*** Marga_ has quit IRC | 19:33 | |
victor_lowther | with vms, at least the hypervisor is auditable (for opensource ones, at least) | 19:33 |
*** andreykurilin_ has quit IRC | 19:33 | |
victor_lowther | by and large, firmware is not. | 19:33 |
victor_lowther | but enough paranoid ranting. | 19:34 |
*** Marga_ has joined #openstack-ironic | 19:37 | |
*** Marga_ has quit IRC | 19:38 | |
*** Marga_ has joined #openstack-ironic | 19:38 | |
*** Marga_ has quit IRC | 19:38 | |
*** shakamunyi has joined #openstack-ironic | 19:38 | |
*** krtaylor has quit IRC | 19:40 | |
*** zz_jgrimm is now known as jgrimm | 19:40 | |
*** jcoufal has quit IRC | 19:44 | |
*** Marga_ has joined #openstack-ironic | 19:51 | |
*** Marga_ has quit IRC | 19:54 | |
*** krtaylor has joined #openstack-ironic | 19:55 | |
*** Marga_ has joined #openstack-ironic | 19:59 | |
*** Masahiro has joined #openstack-ironic | 19:59 | |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: split discoverd.py into 2 modules https://review.openstack.org/140102 | 20:02 |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: consolidate standard plugins in one module https://review.openstack.org/140134 | 20:02 |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: drop features incompatible with Kilo changes https://review.openstack.org/140135 | 20:02 |
openstackgerrit | Dmitry Tantsur proposed stackforge/ironic-discoverd: Refactoring: cap complexity at 15 and coverage at 90 https://review.openstack.org/140136 | 20:02 |
*** Masahiro has quit IRC | 20:04 | |
*** Marga_ has quit IRC | 20:06 | |
*** rushiagr is now known as rushiagr_away | 20:11 | |
*** subscope_ has joined #openstack-ironic | 20:14 | |
*** mrda-away is now known as mrda | 20:16 | |
mrda | Morning Ironic | 20:16 |
dlaube | g'morning mrda | 20:19 |
dlaube | and everyone else for that matter | 20:19 |
mrda | morning dlaube | 20:23 |
*** alexpilotti has quit IRC | 20:27 | |
NobodyCam | mornign mrda dlaube :) | 20:27 |
*** anderbubble has quit IRC | 20:27 | |
mrda | hey NobodyCam | 20:28 |
*** erwan_taf has quit IRC | 20:31 | |
NobodyCam | :) | 20:43 |
*** eghobo has joined #openstack-ironic | 20:52 | |
NobodyCam | brb fresh coffee is ready | 20:53 |
*** dprince has quit IRC | 20:55 | |
*** marcoemorais has quit IRC | 21:03 | |
*** marcoemorais has joined #openstack-ironic | 21:03 | |
openstackgerrit | Merged openstack/ironic-specs: Introduce driver capabilities https://review.openstack.org/128927 | 21:10 |
*** ifarkas has quit IRC | 21:12 | |
*** Marga_ has joined #openstack-ironic | 21:16 | |
jroll | victor_lowther: there are ways to prevent tenants from successfully flashing firmware, jfyi :) | 21:21 |
jroll | 19:20:02 NobodyCam | general question, as an operator are tenants of bare metal nodes allowed to reflash bios/device roms? <- you have to assume they will try, even if they aren't "allowed" to | 21:22 |
victor_lowther | jroll: yes, but the are not widespread. | 21:22 |
*** andreykurilin_ has joined #openstack-ironic | 21:22 | |
jroll | right | 21:23 |
jroll | just throwing that out there | 21:23 |
victor_lowther | Secure hardware allows you to physically disable the erase/reprogram functionality. | 21:25 |
victor_lowther | Especially paranoid hardware (like the yubikey) does not allow flashing at all. | 21:25 |
NobodyCam | brb | 21:26 |
victor_lowther | If you are brave and have the pinouts, you might be able to disable flash capabilities with a magnifying glass and an xacto knife. :) | 21:27 |
jroll | lol | 21:29 |
* JayF updated IPA subteam status on the whiteboard | 21:35 | |
jroll | thanks | 21:37 |
*** subscope_ has quit IRC | 21:48 | |
*** Masahiro has joined #openstack-ironic | 21:48 | |
*** erwan_taf has joined #openstack-ironic | 21:50 | |
*** Masahiro has quit IRC | 21:53 | |
*** igordcard has quit IRC | 21:53 | |
*** igordcard has joined #openstack-ironic | 21:55 | |
*** jgrimm is now known as zz_jgrimm | 21:57 | |
*** zz_jgrimm is now known as jgrimm | 21:58 | |
*** Marga_ has quit IRC | 22:02 | |
*** erwan_taf has quit IRC | 22:04 | |
*** anderbubble has joined #openstack-ironic | 22:04 | |
JayF | adam_g: just advertised your fix for 1398128 in #openstack-qa ... would like to get these intermittant gate failures fixed so I can start back the IPA-voting-on-Ironic conversation | 22:08 |
adam_g | JayF, ah, cool thanks! | 22:12 |
*** igordcard has quit IRC | 22:12 | |
*** Marga_ has joined #openstack-ironic | 22:15 | |
*** Marga_ has quit IRC | 22:15 | |
*** Marga_ has joined #openstack-ironic | 22:22 | |
*** anderbubble has quit IRC | 22:23 | |
*** Marga_ has quit IRC | 22:27 | |
*** Marga_ has joined #openstack-ironic | 22:29 | |
*** andreykurilin_ has quit IRC | 22:30 | |
*** andreykurilin_ has joined #openstack-ironic | 22:31 | |
*** ChuckC has quit IRC | 22:32 | |
*** spandhe has quit IRC | 22:34 | |
*** spandhe has joined #openstack-ironic | 22:35 | |
*** spandhe has quit IRC | 22:37 | |
jroll | JayF: https://review.openstack.org/#/c/139782/ | 22:39 |
*** spandhe has joined #openstack-ironic | 22:39 | |
*** Marga_ has quit IRC | 22:39 | |
*** Marga_ has joined #openstack-ironic | 22:44 | |
*** alexpilotti has joined #openstack-ironic | 22:47 | |
NobodyCam | brb | 22:47 |
*** Marga_ has quit IRC | 22:48 | |
*** vdrok has quit IRC | 22:50 | |
openstackgerrit | Jay Faulkner proposed openstack/ironic-python-agent: Use oslo.utils and oslo.concurrency https://review.openstack.org/138116 | 22:51 |
*** Marga_ has joined #openstack-ironic | 22:59 | |
*** Marga_ has quit IRC | 23:02 | |
*** marcoemorais has quit IRC | 23:07 | |
*** Marga_ has joined #openstack-ironic | 23:07 | |
*** marcoemorais has joined #openstack-ironic | 23:08 | |
*** alexpilotti has quit IRC | 23:11 | |
*** ryanpetrello has quit IRC | 23:12 | |
*** ChuckC has joined #openstack-ironic | 23:14 | |
*** ryanpetrello has joined #openstack-ironic | 23:14 | |
*** ryanpetrello has quit IRC | 23:20 | |
*** marcoemorais1 has joined #openstack-ironic | 23:32 | |
*** marcoemorais has quit IRC | 23:33 | |
*** Masahiro has joined #openstack-ironic | 23:37 | |
*** Masahiro has quit IRC | 23:41 | |
*** jgrimm is now known as zz_jgrimm | 23:41 | |
*** marcoemorais1 has quit IRC | 23:46 | |
*** marcoemorais has joined #openstack-ironic | 23:46 | |
*** eghobo has quit IRC | 23:48 | |
*** marcoemorais has quit IRC | 23:50 | |
*** marcoemorais has joined #openstack-ironic | 23:50 | |
*** spandhe has quit IRC | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!