Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul test suite and GH CI workflow #348

Merged
merged 190 commits into from
Oct 28, 2024
Merged

Overhaul test suite and GH CI workflow #348

merged 190 commits into from
Oct 28, 2024

Conversation

wneessen
Copy link
Owner

@wneessen wneessen commented Oct 26, 2024

This PR improves the overall test suite of go-mail. Most of the tests have been reworked and improved for code readability and maintainability. This PR does not change the go-mail/smtp test suite.

The testing is now more structured and standarized. The coverage is now +90% for all files.
Additionally, the Github CI workflow has also been updated to be more concise by combining all the jobs into one CI workflow file. The CirrusCI workflow has been removed and replaced by FreeBSD github actions.

During the refactoring the following bugs were found and fixed:

  • Add error check for nil SMTP authentication method 1c8b290
  • Add error handling for nil DialContextFunc 35f92f2
  • Set fallbackPort to 0 in SetTLSPortPolicy 3efd2b5
  • Move delivery status update after writer close check 42c6379
  • Remove redundant connection check in auth function 5e3ebcc
  • Remove redundant connection check in send function 45ebcb9
  • Add nil check for smtpClient in checkConn function 4a8ac76
  • Refactor error handling in eml parser 769783f
  • Initialize address header map and enforce single 'From' address 08fe44c
  • Fix ToFromString to handle and trim empty addresses c99b6c3
  • Refine CcFromString to handle spaces and empty addresses 855d7f0
  • Improve BccFromString to handle spaces and empty addresses f079ea0
  • Refactor SetDate to use SetDateWithValue ae15a12
  • Initialize genHeader in RequestMDNTo method 4f97cd8
  • Refactor RequestMDNAddTo 4fe9022
  • Follow upstream for HELO during Quit bug 8353b4b - The reported upstream bug was caused by this PR. See: net/smtp: Quit() will continously fail if a previous EHLO/HELO failed golang/go#70011

The testdata directory entry was incorrect and has been removed. Additionally, an extra newline at the end of the file was removed for consistency.
Switched ErrNoOutWriter from a string constant to an error object for improved error handling consistency. This change enhances clarity and reduces the risk of error handling inconsistencies in the future.
Improved the test suite for the Base64LineBreaker. Tests are now written more consistently. Maintainability and readability have been improved as well.
This commit introduces a new file, logo.svg.base64, into the testdata directory. This encoded SVG logo will be useful for testing purposes.
This change ensures that coverage data files are not tracked by Git. The addition helps keep the repository clean from auto-generated coverage reports.
These files specify the copyright and licensing information for the logo assets. They ensure that our usage of these images complies with the CC-BY-ND-4.0 license.
This commit adds a check for errors when decoding base64 strings in the b64linebreaker_test.go file. If an error occurs, it logs a meaningful error message, improving the test's robustness and clarity.
The previously commented-out empty byte array is now uncommented to include an empty string as part of the test cases. This change ensures that the FuzzBase64LineBreaker function properly handles an empty input scenario.
Ensure that the SMTP authentication method is not nil by adding a corresponding error check in the WithSMTPAuthCustom function. Introduced a new error, ErrSMTPAuthMethodIsNil, to handle this validation.
Reorganized and refactored tests for `NewClient` by adding individual test cases for different client creation scenarios, including various options and validation. Improved clarity and coverage by isolating cases, enhancing error messages, and removing redundancy.
Introduce ErrDialContextFuncIsNil to handle cases where the dial context function is not provided. Update WithDialContextFunc to return this error when a nil function is passed.
Expanded the client test cases to include DSN mail return types and multiple DSN recipient notify types. Also added tests for setting dial context functions using both net.Dialer and tls.Dialer, including error handling for invalid options.
Introduced `context` package for new tests. Added checks for `WithDialContextFunc`, `WithLogAuthData`, and TLS policy assertions. Improved test failure handling and removed obsolete tests.
The default fallback port should be reset to 0 when setting the TLS port policy. This ensures the fallbackPort does not retain an incorrect value when switching policies.
Implemented detailed test cases for various TLS policies and settings, port configurations, and SSL options. Also added tests for debugging functionalities of the Client, enabling robust validation of different scenarios.
Introduced unit tests for Client's SetTLSConfig, SetUsername, and SetPassword methods. The tests cover various scenarios, including setting valid configurations, handling nil inputs, and overriding previous settings. This improves our test coverage and ensures the reliability of these methods.
Introduce a new test to validate the functionality of the Client.SetSMTPAuth method. This test covers various SMTP authentication types and ensures that the method correctly sets the expected authentication type while also verifying the override behavior for different custom and default authentications.
Removed outdated tests for TLSConfig, Username, Password, and SMTPAuth that were commented out. Added a new test for the SetSMTPAuthCustom method, specifically for PLAIN authentication, ensuring proper behavior and type checks.
Replaced the single test case for setting custom SMTP authentication with a table-driven approach. This refactor improves test coverage by including multiple authentication methods such as CRAM-MD5, LOGIN, PLAIN, SCRAM, and XOAUTH2.
Reintroduces and enhances several client tests including `SetLogAuthData`, `Close`, `DialWithContext`, and others. Deprecated and non-functional tests are removed, and a new log parsing method is added for enhanced logging validation.
Consolidate server configuration properties into a new serverProps struct for better code clarity and future extensibility. This change involves updating simpleSMTPServer and test cases to use the new struct, allowing more control over server behavior during tests.
Refactored the test code to initialize buffers using bytes.NewBuffer(nil) instead of &bytes.Buffer{}. This change ensures a consistent and idiomatic initialization of byte buffers throughout the test cases.
Replaced &bytes.Buffer{} with bytes.NewBuffer(nil) for buffer initialization in various client tests. This change ensures more idiomatic and consistent creation of byte buffers throughout the test cases.
Add test cases to validate client behavior on connecting with an invalid host and encountering a HELO failure. This helps ensure the client handles these error scenarios correctly, maintaining robustness and reliability.
Introduce a new test case to ensure the SMTP client fails gracefully when an invalid HELO command is used. This includes validating error handling and maintaining the client connection integrity. Also, optimize EHLO/HELO command handling by enhancing syntax checking and error response generation.
Ensure QUIT command can be sent even if initial HELO fails. Added a check to skip retrying HELO if it already failed, allowing for proper closing of the connection. This prevents potential hangs or errors during connection termination.
Introduce `WithDebugLog()` in client creation for enhanced logging during tests. Correct handling of certain SMTP command errors by replacing `return` with `break` in switch cases for proper loop continuation.
Corrected grammar in a comment to enhance code readability and maintain consistency. This change does not affect the functionality of the `Quit` method.
Extended test cases to include scenarios for STARTTLS and SSL connections. This includes handling TLS configurations, testing certificate handling, and tests for various authentication methods under TLS.
Deleted two test functions for DialWithContext that tested invalid HELO and authentication scenarios. These tests were deemed redundant as the error handling for these cases is covered elsewhere.
Introduce unit tests for EmbedReadSeeker, EmbedHTMLTemplate, EmbedTextTemplate, and EmbedFromEmbedFS methods in the Msg package. These tests cover various use cases and error scenarios, ensuring the correct handling of embedded content.
Eliminated multiple test functions for file attachments and embeds, including TestMsg_AttachFile, TestMsg_AttachFromEmbedFS, and TestMsg_EmbedFile. These tests either duplicated functionality or were no longer relevant to the current code base. Simplifying the test suite enhances maintainability and reduces unnecessary checks.
This test ensures that the Reset method correctly clears all fields in the Msg struct. It verifies that the From, To, Subject, Body, Attachments, and Embeds fields are empty after calling Reset.
Introduced comprehensive test cases for applying middlewares (uppercase and encode) to messages. Also added tests for writing messages to memory buffers and parsing them back. Corrected messages in helper functions for consistency.
Implement a new test for the WriteToSkipMiddleware function to validate its behavior with normal mail parts. Ensure that the function properly skips the specified middleware and writes the message to a buffer correctly.
Implemented tests for the Msg.Write method and enhanced WriteTo error handling. Also added a failReadWriteSeekCloser type to explicitly simulate IO failures in tests, ensuring comprehensive reliability of error handling mechanisms.
Unified the test cases for AttachFile, AttachReader, AttachReadSeeker, EmbedFile, EmbedReader, and EmbedReadSeeker by incorporating the previously separate UNIX-specific tests into the general test files. This change enhances maintainability by reducing code duplication and ensuring more consistent test coverage across platforms.
A newline was added after an anonymous function block to improve code readability. This change adheres to proper formatting practices and enhances the maintainability of the code.
Introduce tests for "AttachFile" with a normal file and "WriteTo" with long headers. Additionally, remove outdated and redundant test functions.
Standardize variables by using expressions for consistency. This change ensures that all environment variables for Unix and Sendmail tests are sourced from GitHub variables.
Replaced bytes.Buffer{} with bytes.NewBuffer(nil) for buffer initialization in NewReader method of Msg. This change enhances clarity and consistency in buffer handling.
Reorganized message writing tests to create more concise and specific scenarios, including new context tests and WriteToFile tests. Removed outdated and redundant test cases to streamline the test suite.
Implemented new tests to validate the Msg.UpdateReader method, ensuring its functionality for both success and failure scenarios. Removed obsolete tests for Msg.NewReader to maintain code clarity and relevance.
Introduce a new `FailTemp` property to simulate a temporary failure during the DATA transmission phase in tests. This helps ensure better coverage and handling of specific error conditions.
Introduce several unit tests for the Msg.HasSendError function to ensure its correct behavior under different conditions like unsent messages, failed deliveries, and temporary errors. This enhances the validation of message error states within the system.
Added a check for the "PERFORM_UNIX_OPEN_WRITE_TESTS" environment variable to conditionally skip tests. This ensures tests are only executed when the variable is set to "true". Additionally, commented out the TestMsg_WriteToTempFileFailed function.
Implemented unit tests for WriteToTempFile, including successful writing to a temporary file and failure cases due to an invalid TMPDIR. These tests ensure the method behaves correctly under different conditions and enhances code robustness.
Added tests for `AttachReader`, `AttachReadSeeker`, `EmbedReader`, and `EmbedReadSeeker` methods with consecutive `WriteTo` calls to ensure attachments are not lost. This addresses issue #110 on GitHub. Also, added tests for `SetBodyWriter` with a nil option and improved existing tests for `hasAlt` and `hasMixed` methods. This commit concludes the tests for msg.go. We have achieved the highest possible coverage.
Reorganize existing tests for msgWriter and writeMsg, adding subtests for various encoding, charset, and failure scenarios. Enhanced tests to cover multipart/mixed, multipart/related, multipart/alternative, application/pgp-encrypted, application/pgp-signature messages, and handling of preformatted headers.
This commit introduces unit tests covering various scenarios for attaching files in email messages using msgWriter. These tests check for proper encoding, custom content types, different transfer encodings, custom descriptions, and handling attachments without body parts.
The TestRcpt constant was defined but never used within the test file, leading to unnecessary clutter. Removing it enhances code readability and maintainability.
Introduce tests to validate MsgWriter's handling of message parts. This includes ensuring charset defaulting behavior and proper inclusion of part descriptions.
Implemented tests to validate the MsgWriter writeString method for successful writes, failure scenarios, and handling pre-existing errors in the writer. This improves coverage and ensures robustness of the MsgWriter component.
Introduce comprehensive test cases for the writeHeader and writeBody functions in msgWriter. These tests cover various scenarios including handling of multiple header values, edge cases, and error conditions in body writing operations. This concludes all tests for msgwriter.go.
Introduced a new test attachment file for testing purposes. Also added a corresponding license file indicating copyright and licensing information for transparency and compliance.
Added platform-specific checks for header and attachment content in `msgwriter_test.go`. This ensures compatibility and correct behavior on different operating systems, such as Windows and FreeBSD.
Updated the base64-encoded string for attachments in the tests to reflect the correct newline character for Windows. This ensures that the test cases correctly validate the presence of attachments in the mail messages.
@wneessen wneessen marked this pull request as ready for review October 28, 2024 21:51
@wneessen wneessen merged commit c8478fb into main Oct 28, 2024
29 checks passed
@wneessen wneessen deleted the feature/overhaul-tests branch November 22, 2024 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant