Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix-rnn-detection #453

Closed
wants to merge 308 commits into from
Closed

Fix-rnn-detection #453

wants to merge 308 commits into from

Conversation

eldraco
Copy link
Collaborator

@eldraco eldraco commented Feb 13, 2024

Fixes Issue

Closes #330

Changes proposed

  • Fix the RNN not working due to keras update (Bidirectional not loaded)
  • Add a new dataset to test the CC
  • Update the RNN model in Slips
  • Add new documentation of RRN and datasets

Check List (Check all the applicable boxes)

  • My code follows the code style of this project.
  • My change requires changes to the documentation.
  • I have updated the documentation accordingly.
  • All new and existing tests passed.
  • This PR does not contain plagiarized content.
  • The title of my pull request is a short description of the requested changes.

@eldraco
Copy link
Collaborator Author

eldraco commented Apr 20, 2024

This is weird, because even without using the module the test fails. So the module is not the problem. Analysing the tests

@eldraco eldraco force-pushed the fix-rnn-detection branch from 282a00d to a053cd3 Compare June 18, 2024 13:23
eldraco added 26 commits June 25, 2024 09:41
@eldraco
rnn_module. Improve the dataset of known CC channels. And add one mor…
30077ae 
…e synthetic CC
@eldraco
rnn_module. Update the RNN ML model to the latest CC channels
a650d64
@eldraco
rnn_module. Improve README for rnn
78db836
@eldraco
documentation. Add new page for Slips datasets
b1c7e41
@eldraco
rnn_module. Add the graphs of loss and accuracy for the latest model
ea98aad
@eldraco
datasets. Add 2 new testing datasets for CC detection.
5a1c66f
@eldraco
rnn_module. Fix error loading Bidirectional model with new keras library
a3e7617
@eldraco
Much better documentation of how the stratosphere letters work and th…
531ef31 
…e RNN model
@eldraco
readme. More changes
4c246be
@eldraco
Update README.md Add correct figure 1
796fe14
@eldraco
rnn. Delete old datasets
414de98
@eldraco
rnn. Delete old datasets
c4aab87
@eldraco
rnn. Rename datasets
1b394a3
@eldraco
rnn. Add a jupyter python notebook to do better training of CC models
162ff57
@eldraco
rnn. Add comments to code
fbdff59
@eldraco
ipynb. Much better code and evaluation and separation of data
27a2123
@eldraco
readme. Better explanations
5ffeb24
@eldraco
readme. Include plots
f7ea264
@eldraco
rnn. Add the model v1
3dcaafa
@eldraco
rnn. Add the model v1.1
d7c8e71
@eldraco
docs. Move the plots to the docs folder
380b446
@eldraco
docs. Add the plots of v1 and v.11 to the docs folder
265532c
@eldraco
rnn. Add the original model
a561c50
@eldraco
rnn. improve jupyter notebook
2eb88ed
@eldraco
rnn. Add optimization with optuna
f439966
@eldraco
Add log of the trials
5d2859b
eldraco and others added 27 commits July 9, 2024 22:00
@eldraco
rnn_module. Add the graphs of loss and accuracy for the latest model
94dd5a9
@eldraco
docs. Move the plots to the docs folder
1c70320
@eldraco
Change the model to a new one optimized with Adam, not RMSProp
8f459e2
@eldraco
Update the RNN model to v1.2
d377d9e
@eldraco
rnn. Make the model v1.4 the default model for rnn
48c6178
@AlyaGomaa @eldraco
dependency_image: use python3.10
c228027
@AlyaGomaa @eldraco
dependency_image: remove tzlocal from requirements
5e06a71
@AlyaGomaa @eldraco
dependency_image: install python3-tzlocal and python3-certifi after i…
005a5fe 
…nstaling python 3.10
@eldraco
readme change
cf6197b 
Signed-off-by: Sebastian Garcia <[email protected]>
@eldraco
readme change
a461bd0 
Signed-off-by: Sebastian Garcia <[email protected]>
@eldraco
Merge branch 'fix-rnn-detection' of github.com:stratosphereips/Strato…
8410bf7 
…sphereLinuxIPS into fix-rnn-detection

Signed-off-by: Sebastian Garcia <[email protected]>

# Conflicts:
#	docker/dependency-image/Dockerfile
#	modules/rnn_cc_detection/README.md
#	modules/rnn_cc_detection/rnn_cc_detection.py
@AlyaGomaa
update requirements.txt and remove most fixed versions
d401216
@AlyaGomaa
update ubuntu dockerfile to use ubuntu22:04 and python 3.10.12
082f030
@eldraco
rnn. Add new model v1.5
20897b9
@eldraco
rnn. Make new model v1.5 the one used by default
5446135
@eldraco
rnn. Add new model v1.5 performance graphs to docs
de8d3d6
@eldraco
rnn. Update the readme
aa5eefc
@eldraco
rnn. Fix bad previous merging
eb842a0
@eldraco
rnn. Training now includes a reshaping since the libraries changed an…
0bbce8c 
…d the code was not woring any more
@eldraco
Merge branch 'develop' into fix-rnn-detection
5826710
@eldraco
Merge branch 'fix-rnn-detection' into alya/migration_to_python3.10.12
d55e787 
Signed-off-by: Sebastian Garcia <[email protected]>

# Conflicts:
#	install/requirements.txt
@eldraco
rnn. Fix error that I put the new model in the wrong place.
32b36c4
@eldraco
rnn. Fix the Dockerfile to run for macos-m1.
39c03da 
Signed-off-by: Sebastian Garcia <[email protected]>
@eldraco
Merge branch 'fix-rnn-detection' of github.com:stratosphereips/Strato…
1eeca31 
…sphereLinuxIPS into fix-rnn-detection
@eldraco
fix-rnn. Delete misplaced model
b213227
@eldraco
Fix the export of stratoletters in case the tuple is none
da53121
@eldraco
rnn. When detecting C&C only take into account the established flows.
ba5e52d
@eldraco
Copy link
Collaborator Author

eldraco commented Jul 16, 2024

This branch was ok, and merged into the new branch alya/migration_to_python3.10.12.
So eventually this code will make it to develop

@eldraco eldraco closed this Jul 16, 2024
@eldraco eldraco deleted the fix-rnn-detection branch July 16, 2024 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@eldraco @AlyaGomaa