Monday, 2016-03-07

*** angdraug has joined #openstack-solar00:44
*** angdraug has quit IRC01:45
-openstackstatus- NOTICE: gerrit is going to be restarted due to bad performance07:24
*** ChanServ changes topic to "gerrit is going to be restarted due to bad performance"07:24
*** ChanServ changes topic to "#openstack-solar"07:27
*** dshulyak has joined #openstack-solar08:05
pigmejhello !09:00
pigmejdshulyak: what about that versioning fixes?09:12
dshulyaki didnt make them09:14
dshulyakhm, actually what versioning fixes?09:16
dshulyakversion in resource?09:16
dshulyakthere wasnt  any problem with that09:16
*** salmon_ has joined #openstack-solar09:19
pigmeji fixed parsing09:22
pigmejprevious 8.0 was valid09:22
salmon_rly? :D09:24
pigmejsalmon_: well, in some parst:P09:28
pigmejwe didn't merge it yet because of f2s09:29
salmon_hmm, so 8.0 is valid or not?09:30
pigmejit is for now, but it shouldn't09:30
pigmejbecause 8.0.0 is valid for semver09:30
salmon_ah, ok. I misunderstood you at the beginning09:31
pigmejsalmon_: coffee!!09:32
salmon_*tea :)09:32
pigmejand see what it did to you...09:37
openstackgerritMerged openstack/solar: Re-visit visited predecessors for react_on events
pigmejguys, can you review changes ?09:39
pigmejwe have pretty a lot not merged stuff...09:39
openstackgerritJedrzej Nowak proposed openstack/solar: Added naive method to check if object already exists
salmon_reviewing now09:42
salmon_astute :P09:42
dshulyakwhat astute?09:48
dshulyakpigmej: ok, i thought that 8.0 is valid, but anyway i changed everything to 8.0.0 for 3-4 days09:49
dshulyak3-4 days ago09:49
dshulyaksalmon_: btw what happened with 3 controllers?09:50
salmon_dshulyak: this one from astute
salmon_dshulyak: 3 controllers failed with error in heat :(09:51
pigmejdshulyak: then we can safely merge this
salmon_dshulyak: but it looks like error in fuel-library again09:51
pigmejsalmon_: "always blame others (tm)"09:52
salmon_pigmej: last time it was an error in libraryr :P09:52
dshulyakmaybe it fails because we dont have retries, and in astute puppet will be retried ateast once09:52
pigmejshouldn't solar o retry "fix" it then?09:53
salmon_dshulyak: I retried manually a few times09:53
dshulyaksolar o retry is manual operation, there should be retries for each task that will be triggered in runtime09:54
dshulyaksalmon_: have u created all 3 at once?09:55
pigmejyeah dshulyak sure09:55
dshulyakthere is quite a lot of problems with updates09:56
salmon_sure we should have it but it's not a case here. I retried it few times and there are retries in the task.09:56
salmon_dshulyak: yes 3 at the same time09:56
*** bogdando has quit IRC10:01
*** bogdando has joined #openstack-solar10:03
openstackgerritMerged openstack/solar: Added systemd solar-worker.service
openstackgerritMaciej Kwiek proposed openstack/solar: Get task start and end time with defaults
mkwieksalmon_: please review this one: , it should help with your KeyError, but I couldn't find the root cause, as I can't even reproduce it. I think we shouldn't really merge such fixes, but I am out of ideas for this one.11:33
pigmejmaybe backend issue?11:34
pigmejafair lukasz uses sqlite11:35
salmon_mkwiek: I agree with dshulyak11:38
pigmejyaeh me too, just wtf with that error then?11:38
mkwiekpigmej: I am also using sqlite11:38
pigmejsalmon_: can you reproduce it somehow?11:39
mkwiekyeah, I agree with you all :D11:39
salmon_pigmej: I don't need to, I'm hiting it all the time :)11:39
mkwieksalmon_: can you send me your database?11:39
pigmejstill we need somehow to figure out the reason for it11:40
salmon_mkwiek: wait a few minutes, I will try to reproduce it and will send you a  db11:41
mkwiekcool, thanks a lot11:41
openstackgerritBogdan Dobrelya proposed openstack/solar: Separate vagrantfile for the docker provider
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
salmon_dshulyak: pigmej how can I fix it
pigmejsalmon_: solar o release_lock {id}12:02
pigmejhow did you lokc it though?12:02
pigmejsolar o release-lock12:03
salmon_pigmej: I saved vm state :)12:03
pigmejok :)12:04
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
dshulyaksalmon_: have u tried to generate graph for 3 controllers? it just hangs for me13:07
dshulyakand it hangs not even in solar.. it is tred part13:08
salmon_dshulyak: sure it worked13:08
dshulyakok, maybe i need to wait13:09
salmon_dshulyak: hmm, It was quite fast. max 1-3s delay13:09
pigmejok 30% speedup with pg gained :D13:32
pigmejok and no longer 100% CPU13:34
pigmejstill checking :)13:34
pigmejto verify :)13:34
pigmejTotal Delta: 13.182318210613:35
pigmejTotal Time: 93.494682073613:36
pigmejwtf ?13:36
pigmejmkwiek: I'm pretty sure that somethign is still wrong in our time calculations13:36
mkwiekdelta should be higher then total time, right?13:38
pigmejshould be but in this example everythign is sequential13:39
pigmejbut everytime I restart this job13:39
pigmejtotal delta is added to total time13:39
pigmejTotal Delta: 14.358031034513:39
pigmejTotal Time: 104.08819508613:39
mkwiekwhat do you mean added?13:39
pigmejTotal Delta: 11.281550884213:40
pigmejTotal Time: 180.40907001513:40
dshulyakpigmej: so u are doing restarts or what?13:40
pigmej"but everytime I restart this job" :)13:40
mkwiekI guess it makes sense?13:40
dshulyakyeah, i just wanted to clarify13:40
pigmejmkwiek: that total time for the same job increases ?13:40
mkwiekbecause other tasks were already finished?13:40
pigmejrestart starts from 013:41
pigmejif I would restart it tomorrow it would say me ~90000 seconds ?13:41
dshulyakthere is still no reset of time on restart13:41
pigmejso total time is time of first task - time of last task ?13:42
pigmejit's total delta, right?13:42
pigmejso what's total time then?13:42
dshulyakno, total time is last - first13:42
dshulyaktotal delta is sum of (end - start)13:42
dshulyakso in your case it should equal13:42
mkwiekdelta is time of all tasks combined, so if you have parallel tasks, you can end up with delta higher than total time13:42
pigmejok, so total delta should be highher than totla time, isn't it?13:42
pigmejmkwiek: so we have a bug somewhere there13:43
mkwiekyeah, I guess so13:43
pigmejbecuase I have total time higher than total delta ;p13:43
dshulyakas i said - there is no reset of start_time on reset13:43
mkwiekyou are on master, right?13:43
dshulyakand end_time13:43
pigmejdshulyak: yup it seems so13:43
dshulyaki somehow broke graphviz with nfs :)13:44
mkwiekpigmej: should I submit it to launchpad, or will you?13:44
mkwiekI will take care of it13:44
pigmejmkwiek: ok I wil ldo it13:45
openstackLaunchpad bug 1554058 in Solar "Total Time > Total Delta" [Undecided,New]13:46
pigmejbeautiful report I know13:46
dshulyak:( no it is not related to nfs13:46
pigmejdshulyak: check if you have recent files,13:47
mkwiekpigmej: at least there is a way to reproduce it :D13:47
pigmejbecuase for me pycs are sometimes broken on nfs13:47
dshulyakit hangs in graphviz programs, tred or dot, either on reads or brk, and it doesnt make any sense13:47
pigmejwtf monday :)13:48
dshulyaksalmon_: what version of graphviz u are using?13:49
pigmejbtw dshulyak what command exactly is not working?13:50
pigmejsolar o dg last ?13:50
dshulyakpigmej: yes, but not in python part, it hangs on tred | dot …13:51
mkwiekpigmej: where can I find resource 'noop'?13:51
salmon_dshulyak: dg commands are not working for me completly. It  just generates dot file and then I'm generating svg on my local computer13:52
mkwiekpigmej: I guess I can write it myself :D13:52
dshulyakmkwiek: u can find one in f2s13:52
dshulyaksalmon_: yes, strange that with 1 controller ( ~100 tasks) it works well13:53
pigmejmkwiek: wait I can give it you13:54
salmon_dshulyak: I have 317 tasks13:54
mkwiekpigmej: cool13:54
pigmejbut well13:54
pigmejthis is the meta :P13:54
mkwiekyeah, I guessed so :D13:55
mkwiek*thans even13:55
dshulyaksalmon_: btw what do you mean by completely? it hangs or what?14:03
salmon_dshulyak: no, just generating dot file14:05
mkwiekpigmej: what do you mean by "then normal solar things"?14:05
mkwieksolar changes stage and process?14:05
mkwiekor sth else?14:05
pigmejstage / process14:05
salmon_I had some problem with installing package for dot/tred commands. I don't remember what was the issue14:06
pigmejdshulyak: ewetyhing works prefectly for me about graphs ...14:06
pigmejboth o dg last or solar connections graph14:06
dshulyakfor me the problems with generting dot graph starts with ~200 tasks14:07
pigmejI will create 500 and I will tell you :)14:08
pigmejworks for me dshulyak ;/14:08
dshulyaki suspect that the problem might be related to amount of edges..14:08
dshulyakit doesnt work for me on local env either..14:09
pigmejsqlite ?14:09
pigmejpg ?14:09
pigmejcan you send me a dump?14:09
mkwiekpigmej: believe it or not, but I cannot reproduce this bug with incrementing total time :D14:09
pigmejpg backend14:10
pigmejI can reproduce it always14:10
pigmejhave you used my script?14:10
dshulyakpigmej: why do u need database?14:10
mkwiekI had clear db in sqlite, I did exactly what you posted in the bug, and after each restart I am getting new value for total_time14:10
pigmejto check if the problem is DB or dot :)14:10
pigmejyo said that dot command hangs14:11
pigmejso it's rather not DB as long as it produces valid output...14:11
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
mkwiekpigmej: I mean, for me it works as it should, I get new total_time every time :/14:20
pigmejI don't14:20
dshulyakpigmej:, can u try to generate png from this dot graph?14:20
pigmejyou used my scrip ?14:20
dshulyakor svg14:20
pigmejdshulyak: sur14:21
dshulyaktred | dot -Tsvg -o system_log.svg14:21
mkwiekpigmej: yes14:21
dshulyakthis is what doesnt work for me14:21
pigmejdshulyak: I have syntax error three14:21
pigmejWarning: /tmp/ syntax error in line 1 near 'igraph'14:21
dshulyakit looks like u dropped 1st character14:22
pigmejso, it uses 100% CPU14:23
pigmejbut maybe it will generate it;d14:23
dshulyakyeah, so it is the same as for me14:24
pigmejit finished14:24
dshulyakcrap )14:24
pigmejit takes like 2minutes14:25
pigmejon my cpu, so...14:25
dshulyakits very strange, it is not that big14:26
pigmejdshulyak: what is your use case for 'exists' ?14:33
dshulyakif exists: return else create14:35
pigmejcan I instead do create_if_not_exists?14:35
pigmejI don't want to expose raw exists14:35
dshulyakyes, sure, but i need to catch DBException14:35
pigmejwith creaet_if_not_exists you would not need to14:36
pigmejyou could have obj = create_if_not_exists(...)14:36
pigmejdshulyak: because this 'exists' stuff is kinda "experimental" I would prefer not expose public api for "exists"14:37
dshulyaksometimes i am calling cr.create and sometimes usual resource14:37
pigmejbecause then we will be not able to do something smarter with it14:37
pigmejah ;/14:37
pigmejcrap ;/14:37
dshulyakthis tred takes long time to compile..14:38
pigmejmaybe with may_exists(resource_name):14:38
pigmej   cr.create(resource_name,...)14:38
pigmejmaybe this may_exists is not that stupid in fact14:39
pigmejit could support multiple arguments...14:39
dshulyakwell, maybe u can leave it as it is, i am already doing res.load except NotFound14:47
dshulyaki will just switch order and exception14:47
pigmejit's not that I'm against .exists() but this current implementation is ...14:48
pigmejok my pool seems working :D15:05
pigmej30-40% boost on DB ops15:05
pigmejusing pg15:05
openstackgerritMaciej Kwiek proposed openstack/solar: Reset times in tasks on graph reset
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
openstackgerritJedrzej Nowak proposed openstack/solar: Implement better and more solar specific pool for PG
dshulyaki think i will have to implement this one , otherwise it is unclear how to handle partial updates of nailgun graph16:01
dshulyakand with this we will be able to re-run (insert in graph) part of already commited resources16:01
pigmejhow much time will it take?16:02
pigmejroughly :)16:02
dshulyakday or two16:03
pigmejdoesn't sound that bad then I think, it's worth to fix it now than later in fact...16:05
pigmejbut is *everything* else from f2s ready enough?16:05
dshulyakwhat do we need to be ready from f2s?16:13
dshulyakplugins are not ready16:15
pigmejbasic d eployment and prboably some LCM stuff16:15
openstackgerritBogdan Dobrelya proposed openstack/solar: Add vbox/qemu packer build for centos7
dshulyakwhat lcm stuff?16:15
pigmejwe need to support some class2 LCM16:15
dshulyakin f2s there is no support for this16:15
dshulyakand wont be actually :)16:16
pigmejI know but we need to select some (one?) use case and provide it16:16
pigmejsalmon_: what is missing in f2s ?16:16
dshulyakbut i think the only way to do it is manually create composer file16:16
pigmejtotally anti solar ;(16:16
dshulyakthere is no way to do it without manual composer file even with all solar features16:17
pigmejto change mysql pw I will need to create separate composer file ?16:17
dshulyaki am talking about moving mysql from one node to another16:17
pigmejah yeah this one is different story :)16:17
dshulyakmysql pw is what? it is some parameter in mysql?16:18
pigmejyeah this one for sure requires some "policy engine like stuff"16:18
pigmejwhich may be just another composer file16:18
pigmejdshulyak: well, when we had the connections it was just a input for mysql_user resource afair16:18
pigmejand that resource was connected to several other resources16:18
pigmejso we need to support these "simple" use cases like change value and all connected resources are properly changed (we had connections for it)16:19
dshulyakthis is all problematic..16:19
pigmejAND we need somehow provide composer file / python file / whatever to show that "datach database" is also possible16:19
pigmejdshulyak: I know, but I spoke with salmon_ some time ago, and he said taht in the end connections *maybe*  can be restored16:19
pigmejwell, if we will not have *any* answer for even simple LCM, let's then stop even thinking about F2S at all, we need to show basic LCM that does not require any new resources (or removing old) and we need to show some class2 (like datach db)16:20
dshulyakf2s is not an answer to lcm, we still need it to bootstrap fuel composition in solar, after that everything should be done using solar api16:21
pigmejsure, I'm not saying about supporting it from Fuel UI :)16:22
pigmejjust after you import it so solar, then all *solar* stuff should work, so basic LCM, and we should have answer for advanced one16:22
*** angdraug has joined #openstack-solar17:45
*** angdraug has quit IRC18:01
*** angdraug has joined #openstack-solar18:19
*** angdraug has quit IRC19:29
openstackgerritMerged openstack/solar: Reset times in tasks on graph reset
*** angdraug has joined #openstack-solar21:17
*** dshulyak has quit IRC21:47
*** angdraug_ has joined #openstack-solar22:02
*** dshulyak has joined #openstack-solar22:03
*** angdraug has quit IRC22:05
*** dshulyak has quit IRC22:08
*** angdraug_ has quit IRC22:42
*** angdraug has joined #openstack-solar22:42

Generated by 2.14.0 by Marius Gedminas - find it at!