Tuesday, 2016-08-09

*** GoceVida has joined #openstack-security00:06
*** JAHoagie has quit IRC00:06
*** markvoelker has joined #openstack-security00:11
*** edmondsw has quit IRC01:04
*** browne has quit IRC01:17
*** sdake has quit IRC01:27
*** sdake has joined #openstack-security01:29
*** amitkqed has quit IRC01:43
*** amitkqed has joined #openstack-security01:43
*** vinaypotluri has quit IRC02:21
openstackgerritHe Qing proposed openstack/anchor: Allow a domain start with a number  https://review.openstack.org/35267202:25
*** knangia has quit IRC02:50
*** GoceVida has quit IRC03:06
*** GoceVida has joined #openstack-security03:08
*** dave-mccowan has quit IRC04:24
*** GoceVida has quit IRC04:40
*** GoceVida has joined #openstack-security04:42
*** JAHoagie has joined #openstack-security04:59
*** rcernin has joined #openstack-security05:21
*** sdake has quit IRC05:26
*** sdake has joined #openstack-security05:29
*** dstufft has quit IRC05:41
*** dstufft has joined #openstack-security05:41
openstackgerritzhangyanxian proposed openstack/bandit: Fix some errors in utils.py & calls.py  https://review.openstack.org/35271205:46
openstackgerritzhangyanxian proposed openstack/bandit: Fix some errors in utils.py & calls.py  https://review.openstack.org/35271205:51
*** vinaypotluri has joined #openstack-security06:01
*** xut_xut has joined #openstack-security06:13
*** sweston has quit IRC06:26
*** sweston has joined #openstack-security06:26
*** sdake has quit IRC06:38
*** pcaruana has joined #openstack-security06:39
*** shohel has joined #openstack-security06:45
*** tesseract- has joined #openstack-security06:45
*** liverpooler has joined #openstack-security06:45
*** JAHoagie has quit IRC07:01
*** browne has joined #openstack-security07:15
*** browne has quit IRC07:15
*** elo has quit IRC07:31
*** sdake has joined #openstack-security07:51
*** xut_xut has left #openstack-security07:51
*** markvoelker has quit IRC07:59
*** sdake has quit IRC08:06
*** markvoelker has joined #openstack-security09:00
*** markvoelker has quit IRC09:05
openstackgerritMerged openstack/bandit: Fix some errors in utils.py & calls.py  https://review.openstack.org/35271209:49
*** markvoelker has joined #openstack-security10:01
*** markvoelker has quit IRC10:05
*** sdake has joined #openstack-security10:53
*** markvoelker has joined #openstack-security11:02
*** shohel has quit IRC11:02
*** markvoelker has quit IRC11:06
openstackgerritDoug Chivers proposed openstack/security-doc: Added templates for security review notes and findings  https://review.openstack.org/35210211:07
*** sdake has quit IRC11:08
*** vinaypotluri has quit IRC11:11
*** markvoelker has joined #openstack-security12:02
*** markvoelker has quit IRC12:07
*** markvoelker has joined #openstack-security12:23
*** dave-mccowan has joined #openstack-security12:39
*** catintheroof has joined #openstack-security12:46
*** jass93 has quit IRC12:48
*** _elmiko is now known as elmiko12:57
*** zul_ has quit IRC13:00
*** zul_ has joined #openstack-security13:06
*** sdake has joined #openstack-security13:09
*** cleong has joined #openstack-security13:12
*** sdake has quit IRC13:17
*** sdake has joined #openstack-security13:19
*** edmondsw has joined #openstack-security13:19
*** sdake_ has joined #openstack-security13:23
*** sdake has quit IRC13:24
*** JAHoagie has joined #openstack-security13:46
openstackgerritDoug Chivers proposed openstack/security-doc: Added templates for security review notes and findings  https://review.openstack.org/35210213:51
*** dikonoor has joined #openstack-security13:53
*** edtubill has joined #openstack-security14:12
*** liverpooler has quit IRC14:19
*** JAHoagie has quit IRC14:20
*** sdake_ is now known as sdake14:30
*** edtubill has quit IRC14:34
*** subscope has joined #openstack-security14:39
*** knangia has joined #openstack-security14:40
*** B_Smith has quit IRC14:43
*** dikonoor has quit IRC14:45
*** catintheroof has quit IRC14:47
*** catintheroof has joined #openstack-security14:48
*** catintheroof has quit IRC14:53
*** diazjf has joined #openstack-security14:55
*** B_Smith has joined #openstack-security14:57
*** edtubill has joined #openstack-security14:59
*** vinaypotluri has joined #openstack-security15:06
*** mvaldes has joined #openstack-security15:20
*** mdong has joined #openstack-security15:21
*** catintheroof has joined #openstack-security15:25
openstackgerritAastha Dixit proposed openstack/syntribos: Implement config loading schema  https://review.openstack.org/35249715:37
*** mdong has quit IRC15:42
*** pcaruana has quit IRC15:48
*** rcernin has quit IRC15:50
*** tesseract- has quit IRC15:50
*** mdong has joined #openstack-security15:58
*** dikonoor has joined #openstack-security16:07
*** diazjf has quit IRC16:15
*** ccneill has joined #openstack-security16:18
*** austin987 has joined #openstack-security16:23
*** dikonoor has quit IRC16:35
*** JAHoagie has joined #openstack-security16:36
*** jmckind has joined #openstack-security16:37
openstackgerritMerged openstack/syntribos: Logger not registered bug fixed  https://review.openstack.org/35216216:45
*** subscope has quit IRC16:51
*** mvaldes has quit IRC17:28
*** zul_ has quit IRC17:30
*** zul has joined #openstack-security17:34
openstackgerritVinay Potluri proposed openstack/syntribos: Overwriting config options from CLI  https://review.openstack.org/35303917:39
*** zul has quit IRC17:40
*** diazjf has joined #openstack-security17:42
*** pcaruana has joined #openstack-security17:44
openstackgerritVinay Potluri proposed openstack/syntribos: Overwriting config options from CLI  https://review.openstack.org/35303917:45
openstackgerritRahul U Nair proposed openstack/syntribos: Adding sub commands to Syntribos  https://review.openstack.org/35032517:49
*** zul has joined #openstack-security17:52
*** liverpooler has joined #openstack-security17:53
*** nkinder has quit IRC17:53
*** browne has joined #openstack-security17:53
*** mvaldes has joined #openstack-security18:03
*** nkinder has joined #openstack-security18:04
*** liverpooler has quit IRC18:08
*** rcernin has joined #openstack-security18:51
*** diazjf has quit IRC19:00
*** browne has quit IRC19:04
*** catintheroof has quit IRC19:12
*** kragniz has joined #openstack-security19:15
*** diazjf has joined #openstack-security19:31
*** MARIAVICTORIA-MM has joined #openstack-security19:37
*** MARIAVICTORIA-MM has quit IRC19:44
*** MARIAVICTORIA-MM has joined #openstack-security19:46
MARIAVICTORIA-MMhola19:49
*** austin987 has quit IRC19:52
MARIAVICTORIA-MMhola como estan19:52
MARIAVICTORIA-MMholaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa19:55
*** diazjf has quit IRC19:56
*** diazjf has joined #openstack-security19:58
mdongccneill, unrahul: I was just looking through the changes that merged last week, and I just a minor question20:00
ccneillyep20:00
mdongso https://review.openstack.org/#/c/345286/20/syntribos/runner.py@17920:01
mdongline 179, in this CR, we’ve turned the generator into a list20:01
mdongfor the sake of counting them, it seems like20:01
MARIAVICTORIA-MMhola diazjf20:02
MARIAVICTORIA-MMhola20:02
MARIAVICTORIA-MMque hacen20:02
ccneillmdong: yep..20:02
unrahulYup.. The progress bar20:02
mdongthe point of having get_test_cases yield a generator is so that we don’t actually have all the test cases in memory20:02
unrahulHad an issue otherwise20:02
mdongbut now calling list() on it puts all the tests into memory, doesn20:02
mdongdoesn’t it?20:02
ccneillI believe it would20:02
unrahulBut  the list is not that big ryt..20:02
unrahulTo have any sort of memory issues20:02
ccneillwe should see if we can profile the memory usage20:03
ccneillI didn't have much luck using pycallgraph to do that20:03
mdongright, in practice it’s not like I’m noticing any slowdowns or anything, but the original design intent was to avoid that specifically20:03
ccneillbut I don't think we should keep trying to solve for this supposed memory consumption problem when we don't know that it's actually a problem20:03
ccneillso if it's a drastic difference we should figure out how to solve for it, but if it's negligible, I'm okay with letting the whole generator approach go20:04
unrahulI ran it several times and the memory consumption /performance degradation was not showing up. M20:04
unrahulAnd if we check the size of the list returned it shouldn't have much of an impact20:05
MARIAVICTORIA-MMhelp20:05
MARIAVICTORIA-MM:-X20:05
*** MARIAVICTORIA-MM has left #openstack-security20:05
ccneillat the same time, I'm not sure that we should just create this list on the fly for the sole purpose of checking its length.. if we're going to make a list of tests, we might as well do it purposefully20:06
ccneill¯\_(ツ)_/¯20:06
ccneillanyone have experience with any python memory profilers?20:07
mdongunrahul: is that list used for anything except counting?20:07
mdongI dont’ have any experience myself, unfortunately20:07
unrahulNot just for length,  we are using that list to iterate through the tests20:08
unrahulSo replacing the generator for the list20:08
ccneillI have a feeling we're going to have to replace the generator approach if we ever want to do multithreading20:08
ccneillso it's probably worthwhile for us to figure out what the respective memort footprints are anyway20:08
ccneillmemory*20:08
mdongI was under the impression that generators can be shared across threads20:09
ccneillI'm actually not sure on that one..20:09
ccneillbut I bet Nathan would know :D20:09
mdongregardless, yeah, finding out the memory footprint would be valuable20:09
mdonglol yeah, though he’d have strong opinions about the generator for sure lol20:10
ccneillyeah..20:10
unrahulYeah.. +120:10
ccneillI'll look into it20:10
openstackgerritAastha Dixit proposed openstack/syntribos: Implement config loading schema  https://review.openstack.org/35249720:11
mdongso the list is generated only once per test type right? so the maximum size the list can be is (number of payload strings) * (size of testcase class)20:12
ccneillI *think* that's right20:13
mdongthe biggest file in our data folder is os-cmd-execution.txt at 1200 lines, which we don’t actually use for anything…20:14
ccneilloh actually we have list_of_tests and test_cases20:14
ccneillnvm20:15
ccneilltest_cases should be the list of all tests cases for one endpoint, and would (maybe) get garbage collected after each iteration of the loop20:15
ccneillI'll see what I can come up with in terms of profiling the difference20:15
ccneillbrb20:15
*** singlethink has joined #openstack-security20:16
mdongyeah, rough mental math tells me that the list shouldn’t get too big to cause any problems, unless someone decides to pass in a gigantic data file, which isn’t necessarily out of the question20:17
unrahuli liked the statement "os-cmd-execution.txt at 1200 lines, which we dont use" .. hehe..20:24
mdonglol yeah, we really dont use most of whats in our data folder20:25
unrahulhehe..yeah.. i think we need to clean it up..20:25
unrahuli think ccneill  had a card up on trello for cleaning up the data folder..20:31
ccneillunrahul: actually I don't know if we have a trello card for it yet20:42
ccneillbut we probably should20:42
unrahuloh... i remember.. somewhere seeing something like that.. with cleaning up data folder.. may be u had mentioned it in our meetings ... hmmm..20:43
unrahulcan't remember.20:43
*** diazjf1 has joined #openstack-security20:43
*** diazjf has quit IRC20:45
ccneillhttps://trello.com/c/HOGEpKYW/117-sectest-syn-clean-up-revise-files-outside-codebase-docs-etc20:48
ccneilljust added a few things there20:48
ccneillmight rename this card "preparation for 0.5" or something since it encompasses most of what we were talking about earlier for the 0.5 release (accurate docs, no extra cruft, etc.)20:50
*** edtubill has quit IRC20:51
*** MARIAVICTORIA-MM has joined #openstack-security20:52
MARIAVICTORIA-MMhola20:55
*** MARIAVICTORIA-MM has left #openstack-security20:55
ccneillboom! just closed out our "Remove OpenCAFE" card :D20:56
ccneillhttps://trello.com/c/jH4gDppe/27-sectest-syn-remove-opencafe-from-syntribos20:56
*** diazjf1 has quit IRC21:00
unrahul:D .. good bye old friend, aka OpenCAFE21:00
openstackgerritRahul U Nair proposed openstack/syntribos: Standardizing the way we diff signals  https://review.openstack.org/34940321:11
vinaypotluriccneill:  mdong21:15
mdongwhats up?21:15
ccneillsup21:15
vinaypotluriccneill:  mdong    i'm trying to overwrite the existing config values from cmd line but it reads the config values from the file and then the values of the variables change21:16
vinaypotlurihttps://review.openstack.org/#/c/353039/21:16
vinaypotlurii used CONF.set_override method to override the values but not sure how to go ahead with it21:17
ccneillvinaypotluri: so first, I don't think we want to create a method for EVERY override21:17
ccneillit should be more generalized21:17
vinaypotluriok21:18
ccneillI think to start with21:18
ccneillwe should support overriding each of the "syntribos" options (those in list_syntribos_opts)21:18
ccneillinstead of return [all the cli opts], you can make it a list of options21:19
ccneillthen merge it with the list from list_syntribos_opts21:19
ccneillsince they're in different namespaces they shouldn't clash21:19
vinaypotluriok21:20
ccneillthen you can have a method that goes through each opt in list_syntribos_opts, sees if it's defined in the DEFAULT namespace, and override if so21:20
ccneill(same name, but looking at CONF.___ vs. CONF.syntribos.___)21:21
ccneillmake sense?21:21
vinaypotluricool21:21
vinaypotlurigot it21:21
ccneillcool, let me know if you have any other questions or if you need a code review21:22
vinaypotlurialso when i try to overwrite the values it first takes the values from the config file and then overwrites21:23
vinaypotluriis there anything i can do to force to it read the overwritten values21:24
*** diazjf has joined #openstack-security21:26
*** diazjf has quit IRC21:28
*** zigo has quit IRC21:32
*** zigo has joined #openstack-security21:35
*** cleong has quit IRC21:36
ccneillhmmm21:37
ccneillso you mean like make it skip the part where it reads the values from the config file?21:38
ccneillI don't think that's necessary21:38
ccneillhmm.. I guess it might be a problem if you don't want to specify it in the config file, but only want to define it on the command line21:39
ccneillyou might be able to handle that in syntribos.config.handle_config_exception21:39
ccneillso if it complains that it didn't find a value in the config file, you can check and see if it's specified in the command line opts21:41
ccneillhopefully that doesn't require parsing sys.argv[1:]...21:41
*** jmckind has quit IRC21:41
vinaypotluriok21:42
vinaypotluriwill do that21:43
*** mvaldes has quit IRC21:45
*** sdake has quit IRC21:57
*** rcernin has quit IRC22:02
*** jass93 has joined #openstack-security22:03
*** dave-mccowan has quit IRC22:08
*** sdake has joined #openstack-security22:30
*** sdake has quit IRC22:30
*** sdake has joined #openstack-security22:30
*** sdake_ has joined #openstack-security22:33
*** sdake has quit IRC22:34
*** edmondsw has quit IRC22:36
*** sdake_ is now known as sdake22:38
*** singlethink has quit IRC22:40
*** browne has joined #openstack-security22:43
*** mdong has quit IRC22:53
*** mdong has joined #openstack-security23:07
unrahulHey ccneill mdong any interesting presentations from #DEFCON ?23:23
unrahulthat we should check out.?23:23
mdongoh man, there’s lots, though the craziest one I saw was “How to overthrow a government"23:23
mdongI don’t know if it’s up anywhere23:23
ccneillthey haven't put them on youtube yet23:24
ccneillhttps://www.youtube.com/user/DEFCONConference/videos23:24
ccneillbut that's where they'll be when they are posted23:24
unrahulhow to overthrow a govt.. that seems interesting ..hmm..23:25
unrahulyeah.. waiting for them to upload.. i guess by the week end..23:25
mdongtook em a few months to upload them last time, I think23:25
ccneillhttps://www.defcon.org/html/defcon-24/dc-24-news.html#dc24cdtorrents23:26
unrahulwhoa!23:26
unrahulthat long.23:26
ccneillwell, they sell them to companies first lol23:27
unrahulhehe23:27
mdongspeaking of, did we buy the usb drive?23:27
unrahulthat makes sense..23:27
ccneill:X I hope so23:27
ccneillwe have the last 2 years23:27
unrahulIn the CTF there was that AI from CMU competing ryt, u guys know how the team did?23:28
unrahulusb drive..? of the slides and stuff.?23:28
mdongof the videos, the slides are already up somewhere23:28
mdongat one point the AI team was leading, but I think they ended up near the bottom23:28
ccneillhttps://techcrunch.com/2016/08/05/carnegie-mellons-mayhem-ai-takes-home-2-million-from-darpas-cyber-grand-challenge/23:29
mdongah, then the other AI was leading for a bit before finishing near last, idk how CMU’s did23:29
ccneillthey're probably not TOO sad at the CTF loss lol23:29
mdongbut the other AI had humans working too , they just let CMU’s run on its own23:30
unrahulrofl23:30
unrahulyeah , that they got the 2 mil prize..23:30
unrahulwhoa!>.23:30
unrahulit would be really cool how they even do that ryt..23:30
unrahuljust saw this paper https://users.ece.cmu.edu/~arebert/papers/mayhem-oakland-12.pdf23:34
unrahulsome light reading for the evening.23:34
ccneillhaha yeah23:35
ccneillpretty crazy stuff23:35
openstackgerritRahul U Nair proposed openstack/syntribos: Standardizing the way we diff signals  https://review.openstack.org/34940323:38
*** sdake has quit IRC23:45

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!