tests: posix: common: separate into smaller testsuites

[cfriedt]

Is your enhancement proposal related to a problem? Please describe.
The posix.common testsuite contains a lot of mostly unrelated testsuites. As a result, the code size is large, but mostly the data size is extremely large.

It would require less variation from default settings, and would reduce ram requirements to split that testsuite into smaller groups of more closely related testsuites.

Describe the solution you'd like

The obvious way to group related testsuites is by Option Group.

E.g.

• POSIX_BARRIERS - tests: posix: common: split posix_barriers tests into standalone test #81487
• POSIX_C_LIB_EXT - tests: posix: common: split c_lib_ext tests into standalone test #81488
• POSIX_RW_LOCKS - tests: posix: common: split rwlock tests into a standalone test #80943
• POSIX_SEMAPHORES - tests: posix: common: split semaphore tests into a standalone test #80964
• POSIX_SIGNALS - tests: posix: common: split signal tests into a standalone test #80930
• POSIX_SINGLE_PROCESS - tests: posix: common: split single process into standalone test #80981
• POSIX_SPIN_LOCKS - tests: posix: common: split spinlock tests into a standalone test #80965
• POSIX_SYSTEM_DATABASE - tests: posix: common: split POSIX_SYSTEM_DATABASE into a standalone test #81489
• POSIX_THREADS_BASE - tests: posix: common: split posix_threads_base tests into standalone test #81491
• POSIX_THREADS_EXT - tests: posix: common: split posix_threads_ext tests into standalone test #81490
• POSIX_TIMERS - tests: posix: common: split timer tests into a standalone test #80807
• XSI_REALTIME - tests: posix: common: split XSI_REALTIME tests to standalone testsuite #83634
• XSI_STREAMS - tests: posix: common: split xsi_streams tests into standalone test #81492
• XSI_SYSTEM_LOGGING - tests: posix: common: split syslog tests into standalone test #80986
• XSI_THREADS_EXT - tests: posix: common: split xsi_threads_ext into separate test #81741

and potentially others.

testcase.yaml permutations can be designed to enable or disable features that affect different Option Groups, like TSS, TSA, TSH (see Codes).

Additionally, such a grouping would theoretically improve bug attribution capabilities; if an error was traced (either via stacktrace or bug description) to a certain posix function, there would be a 1:1 correspondance to an Option Group, so typically, only that testsuite would need to be executed.

Describe alternatives you've considered
The way the testsuite has been designed historically.

Additional context
This came up in the context of #79443 but it was already thought of as a nice-to-have.

[zacck]

Can I take a stab at this one please, if it's ok could it be assigned to me?

[cfriedt]

@zacck - please do!

[zacck]

@cfriedt I'm new to Zephyr and while I have looked at the links and docs supplied above I believe I need an example to start breaking this down would you be able to assist with some information that would help here?

[ycsin]

@zacck If you have just started with Zephyr, I think you will need to be familiarized with Twister and the different KV pairs in the testcase.yaml, see Test Runner (Twister)

@cfriedt do you expect the zephyr/tests/posix/common test to be broken down into multiple tests? Because while we can break the test into multiple sub-tests within the tests/posix/common/testcase.yaml, the existing testcase.yaml would be pretty big when we want to make sure that each sub-test are tested with different configurations (different C libs, spin validation, tls, ...) that we have: (<number of option group sub-test> x <number of configurations>)

[cfriedt]

@cfriedt do you expect the zephyr/tests/posix/common test to be broken down into multiple tests?

They should be broken down into separate app directories.

That's mainly because the single application is getting quite large, and the configuration requirements are not always clear.

It would be better if the prj.conf and memory requirements were minimal.

[cfriedt]

It could be good to split this into smaller commits / a task list.

[zacck]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

[cfriedt]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

Cool - it can be separate PRs, which should simplify things. I'll do pthreads, because it is likely the most confusing.

[zacck]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

Cool - it can be separate PRs, which should simplify things. I'll do pthteads, because it is likely the most confusing.

It might be more beneficial if you teach me how to do the most confusing parts, that way I and others can grow into being able to help with meatier stuff and contributors like you can get more done since you can effectively direct us.

I do understand that is would be slower and cumbersome, but do consider it enables us to help more.