Code

Memory profile#388

Code

Merged

AlyaGomaa merged 85 commits intostratosphereips:developstratosphereips/StratosphereLinuxIPS:developfrom danieltherealyang:memory-profiledanieltherealyang/StratosphereLinuxIPS:memory-profileCopy head branch name to clipboardSep 15, 2023

+1,822-270

Commits

Commits on Jun 30, 2023

• Merge pull request stratosphereips#375 from stratosphereips/develop

  Show description for 87c52b6
  AlyaGomaa

  authored
  87c52b6

Commits on Jul 11, 2023

• Update README.md

  Show description for 1b5d4a8
  verovaleros

  authored
  1b5d4a8

Commits on Jul 13, 2023

• Squashed commit of the following:

  Show description for 5408718
  danieltherealyang

  committed
  5408718

• done with dev mode

  danieltherealyang

  committed
  77451b2

Commits on Jul 20, 2023

• singleprocess done, need to add autospawned TUI so doesn't block process

  danieltherealyang

  committed
  38eb0e2

Commits on Jul 24, 2023

• prototype multiprocess profiling structure (untested)

  danieltherealyang

  committed
  352a2e9

• changed Dict Type

  danieltherealyang

  committed
  2f0ae56

Commits on Jul 25, 2023

• stdout to pipe file fixed

  danieltherealyang

  committed
  1ab81ac

• non blocking single process

  danieltherealyang

  committed
  36263ac

• signal handling and global pid->process mapping

  danieltherealyang

  committed
  701e422

Commits on Jul 26, 2023

• proc mapping finished

  danieltherealyang

  committed
  1b8b6e2

• moved mutex locking to start and end tracker

  danieltherealyang

  committed
  f968977

• communication works, need to find out why tracer isn't starting on signal

  danieltherealyang

  committed
  3a0de45

Commits on Jul 27, 2023

• memory profiler done

  danieltherealyang

  committed
  e37be2c

• Added notes to memory_profiler_example

  danieltherealyang

  committed
  1e8e976

Commits on Aug 2, 2023

• testing print statements

  danieltherealyang

  committed
  7d2fe86

• removed custom join and terminate

  danieltherealyang

  committed
  eda609c

• wrote documentation

  danieltherealyang

  committed
  0429dad

Commits on Aug 12, 2023

• Added notes on multiprocess live profiling

  danieltherealyang

  committed
  80adab2

Commits on Aug 13, 2023

• memory profiler works correctly with multiprocessing but modules getting started differently. Uncomment line to see.

  danieltherealyang

  committed
  401945f

Commits on Aug 17, 2023

• Summary: fixed inheritance issues preventing modules from getting profiled.

  Show description for 653c398
  danieltherealyang

  committed
  653c398

Commits on Aug 18, 2023

• commented out multiproc_test in slips.py, text from multiprocessPatchMeta kept for debuggin

  danieltherealyang

  committed
  4cd6468

• made changes

  danieltherealyang

  committed
  e76bd3d

• removed print

  danieltherealyang

  committed
  b29cfbd

Commits on Aug 22, 2023

• made small change

  danieltherealyang

  committed
  15fa16c

Commits on Aug 25, 2023

• add logic to calc aid

  AlyaGomaa

  committed
  e5c3c6d

• get aid instead of cid in sqlite db

  AlyaGomaa

  committed
  d696fd6

Commits on Aug 28, 2023

• Merge pull request stratosphereips#384 from stratosphereips/verovaleros-patch-README

  Show description for eb944f5
  AlyaGomaa

  authored
  eb944f5

• Merge pull request stratosphereips#390 from stratosphereips/alya/calc-aid

  Show description for e9664aa
  AlyaGomaa

  authored
  e9664aa

Commits on Aug 29, 2023

• use transactions and try max 2 times to re-execute failed queries before discaring them

  AlyaGomaa

  committed
  05e7754

• Merge pull request stratosphereips#391 from stratosphereips/alya-fix-malformed-sqlite

  Show description for 857736e
  AlyaGomaa

  authored
  857736e

Commits on Aug 30, 2023

• evidence: pass all alert details to sqlite

  AlyaGomaa

  committed
  31167a5

• sqlite: store all alerts details in the alerts table

  AlyaGomaa

  committed
  2db32a7

• sqlite: store twid start and end date in the alerts table

  AlyaGomaa

  committed
  9eb6bbe

• sqlite: init the trial var to keep track of failed sql queries [skip ci]

  AlyaGomaa

  committed
  c5fc002

• sqlite: print sqlite db errors to errors.log

  AlyaGomaa

  committed
  26ec1eb

• sqlite: store alert time in unix format in alerts table

  AlyaGomaa

  committed
  d74f07a

• Merge pull request stratosphereips#392 from stratosphereips/alya-add-sqlite-alerts-table

  Show description for 7dd87ef
  AlyaGomaa

  authored
  7dd87ef

Commits on Aug 31, 2023

• fix unable to get daddr from conn.log flow in flowalerts (quick-fix) - [skip ci]

  AlyaGomaa

  committed
  f36318c

Commits on Sep 4, 2023

• sqlite: make it possible to have multiple instances of the sqlite class per run

  AlyaGomaa

  committed
  b3b68c4

• start a new instance of sqlite for each module in slips

  AlyaGomaa

  committed
  8038295

• start a new instance of sqlite for each Core file in slips

  AlyaGomaa

  committed
  f207f65

• sqlite: create the db if it doesn't exist, or connect to it if it does

  AlyaGomaa

  committed
  1c9045b

Commits on Sep 5, 2023

• make sure the ts used in calculating aid has microseconds

  AlyaGomaa

  committed
  16986a1

• Merge pull request stratosphereips#394 from stratosphereips/alya/assert_microseconds_in_AID_hash

  Show description for 3f5526f
  AlyaGomaa

  authored
  3f5526f

Commits on Sep 6, 2023

• change how we instantiate dbmanager to be able to mock the sqlite db

  AlyaGomaa

  committed
  cdfb448

• change the creation of all modules in the unit tests to mock sqlite and rdb

  AlyaGomaa

  committed
  99fdeb3

• use mock_rdb instead of generic moc_db in all unit tests

  AlyaGomaa

  committed
  10d634b

• don't connect manually to redis in the test_add_flow_to_profile test

  AlyaGomaa

  committed
  0f8e88d

• Merge pull request stratosphereips#393 from stratosphereips/alya/create_many_sqlite_conns_instead_of_1

  Show description for b44b585
  AlyaGomaa

  authored
  b44b585

Commits on Sep 7, 2023

• don't treat dbmanager as a singelton

  AlyaGomaa

  committed
  e34a186

• close the db in the destructor of each class and core file

  AlyaGomaa

  committed
  af07694

Commits on Sep 8, 2023

• make sure the aid stored in the flows.sqlite db is string not bytes

  AlyaGomaa

  committed
  791638d

Commits on Sep 11, 2023

• Merge pull request stratosphereips#395 from stratosphereips/alya/fix_sharing_sqlite_conn

  Show description for ce9bf55
  AlyaGomaa

  authored
  ce9bf55

Commits on Sep 12, 2023

• use a lock even when creating tables

  AlyaGomaa

  committed
  57f1386

• wait 5 seconds before next trial when db is locked

  AlyaGomaa

  committed
  6a42ee3

• commit even on SELECT

  AlyaGomaa

  committed
  2d5a138

• db: wait for the db to beunlocked 20s instead of the default 5s

  AlyaGomaa

  committed
  d1a9132

Commits on Sep 13, 2023

• Merge pull request stratosphereips#396 from stratosphereips/alya/use_lock_even_when_creating_tables

  Show description for d963514
  AlyaGomaa

  authored
  d963514

Commits on Sep 15, 2023

• Squashed commit of the following:

  Show description for 4da65a9
  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  4da65a9

• done with dev mode

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  230ccd9

• singleprocess done, need to add autospawned TUI so doesn't block process

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  d4a485c

• prototype multiprocess profiling structure (untested)

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  4727834

• changed Dict Type

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  eb59aa6

• stdout to pipe file fixed

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  3eba22f

• non blocking single process

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  fb96463

• signal handling and global pid->process mapping

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  af5d04d

• proc mapping finished

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  8b47542

• moved mutex locking to start and end tracker

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  b3ade23

• communication works, need to find out why tracer isn't starting on signal

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  dd4cd25

• memory profiler done

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  8e423cd

• Added notes to memory_profiler_example

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  6dcb63e

• testing print statements

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  3140a1d

• removed custom join and terminate

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  4995eed

• wrote documentation

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  b9da555

• Added notes on multiprocess live profiling

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  13435b5

• memory profiler works correctly with multiprocessing but modules getting started differently. Uncomment line to see.

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  ec0ba70

• Summary: fixed inheritance issues preventing modules from getting profiled.

  Show description for 723acc6
  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  723acc6

• commented out multiproc_test in slips.py, text from multiprocessPatchMeta kept for debuggin

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  42303cb

• made changes

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  9ce6349

• removed print

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  3c339b5

• made small change

  

  danieltherealyang

  authored and

  AlyaGomaa

  committed
  1ff3aa7

• Merge remote-tracking branch 'danieltherealyang/memory-profile' into memory-profile

  AlyaGomaa

  committed
  2bb153c

• add docs for memory and CPU profiler [skip ci]

  AlyaGomaa

  committed
  947be4f

• disable memory profiler by default

  AlyaGomaa

  committed
  cd73bc7