Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ncftp client to basic vm config... might fix some flow tests. #1249

Merged
merged 1 commit into from
Oct 7, 2024

Conversation

petersilva
Copy link
Contributor

seeing in logs #1248

many flow tests failing 1 test (subscribe_ftp...) and I think it's just that the ncftp accellerator binary is not being installed (maybe it used to be included?) anyways... fixes tests failing my test configuration.

sample before output:

NB retries for subscribe amqp_f30 0
NB retries for sender 0

ERROR Summary:
  3824  subscribe_ftp_f70  (1 file)  [ERROR] sarracenia.flow do_download gave up downloading for now, appending to retry queue
  2840  cpump_xvan_f14     (2 file)  [ERROR] v03 end of parse msg.datestamp: 20241005142033.15450698
  65    post_t_dd1_f00     (9 file)  [ERROR] sarracenia.config check_undeclared_options subscribe/q_f71.conf:12 undeclared option:
  1     subscribe_cp_f61   (1 file)  [ERROR] sarracenia.flowcb.v2wrapper __init__ v2 plugin conversion required, do_download too d
  1     watch_f40          (1 file)  [ERROR] 2004966 sarracenia.config.subscription read failed to read to /home/peter/.cache/sr3/

WARNING Summary:
  1912  subscribe_ftp_f70  (1 file)  [WARNING] sarracenia.flow do_download downloading again, attempt 2
  614   cpump_xvan_f14     (1 file)  [WARNING] cache problem : 20200105 
  2     poll_sftp_f62      (3 file)  [WARNING] sarracenia.flow.poll __init__ nodupe_ttl < fileAgeMax means some files could age ou
  2     cpump_xvan_f14     (1 file)  [WARNING] cache problem : reject 
  1     cpump_xvan_f14     (1 file)  [WARNING] cache problem : Yz 
  1     cpump_xvan_f14     (1 file)  [WARNING] cache problem : _t 

	Maximum of the shovels is: 1419

		TEST RESULTS

test 1 success: Log perms confirmed
                 | content of subdirs of /home/peter/sarra_devdocroot |
test 2 success: compare contents of downloaded_by_sub_amqp and downloaded_by_sub_cp are the same
test 3 success: compare contents of downloaded_by_sub_cp and downloaded_by_sub_rabbitmqtt are the same
test 4 success: compare contents of downloaded_by_sub_rabbitmqtt and downloaded_by_sub_u are the same
test 5 success: compare contents of downloaded_by_sub_amqp and recd_by_srpoll_test1 are the same
test 6 success: compare contents of downloaded_by_sub_u and posted_by_shim are the same
test 7 success: compare contents of downloaded_by_sub_amqp and linked_by_shim are the same
test 8 success: compare contents of posted_by_shim and sent_by_tsource2send are the same
test 9 success: compare contents of downloaded_by_sub_amqp and cfile are the same
test 10 success: compare contents of cfile and cfr are the same
broker state:
                 | dd.weather routing |
test 11 success: post	 count of posted files (1419) should be same those in the static data directory	 (1418)
test 12 success: post	 count of rejected files (10) should be same those in the static data directory	 (10)
test 13 success: post	 (1419) t_dd1 should have the same number of items as t_dd2	 (1419)
test 14 success: sarra	 (2838) should receive the same number of items as both post	 (2838)
test 15 success: sarra	 (1419) should publish the same number of items as one post	 (1419)
test 16 success: sarra	 (1419) should winnow the same number of items as one post	 (1419)
test 17 success: subscribe	 (1419) should rx the same number of items as sarra published	 (1419)
                 | watch      routing |
test 18 success: watch		 (1420) should be the same as subscribe amqp_f30		  (1419)
test 19 success: sender		 (1420) should publish the same number of items as watch  (1420)
test 20 success: rabbitmqtt		 (1420) should download same number of items as watch  (1420)
test 21 success: subscribe u_sftp_f60 (1420) should download same number of items as sender (1420)
test 22 success: subscribe cp_f61	 (1420) should download same number of items as sender (1420)
                 | poll       routing |
test 23 success: poll sftp_f62	 (1418) should publish same number of items of sender sent	 (1420)
test 24 success: poll sftp_f63	 (1418) should see the same number of items as poll sftp_f62 posted	 (1420)
test 25 success: subscribe q_f71	 (1418) should download same number of items as poll test1_f62 (1418)
                 | flow_post  routing |
test 26 success: post test2_f61	 (1111) should have the same number of files of sender 	 (1111)
test 27 FAILURE: subscribe ftp_f70	 (506) should have the same number of items as post test2_f61 (1111)
test 28 success: post test2_f61	 (1111) should post about the same number of files as shim_f63	 (1111)
test 29 success: post test2_f61	 (1111) should post about the same number of links as shim_f63	 (1111)
test 30 success: static tree	 (307) directories should be posted twice: for 1st copy and linked_dir by shim_f63	 (614)
                 | py infos   routing |
test 31 success: 0 messages received that we don't know what happened.
                 | C          routing |
test 32 success: cpost both pelles should post the same number of messages (1420) (1420)
test 33 success: cdnld_f21 subscribe downloaded (1111) the same number of files that was published by both van_14 and van_15 (1111)
test 34 success: veille_f34 should post as many files (1111) as subscribe cdnld_f21 downloaded (1111)
test 35 success: veille_f34 should post as many files (1111) as subscribe cfile_f44 downloaded (1111)
test 36 success: 0 there should be no unacknowledged messages left, but there are 0
test 37 success: 0 there should be no messages ready to be consumed but there are 0
test 38 success: there should be 1 process in wVip state
Overall static_flow 37 of 38 passed (sample size: 1418) !

found the message in the log, added it, re-ran and got 38/38.

@petersilva petersilva marked this pull request as ready for review October 7, 2024 13:35
@petersilva petersilva merged commit 26799fc into development Oct 7, 2024
25 of 57 checks passed
@petersilva petersilva deleted the issue1240 branch December 13, 2024 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants