[REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7570

editorialbot · 2024-12-04T20:56:49Z

Submitting author: @dylanbouchard (Dylan Bouchard)
Repository: https://github.com/cvs-health/langfair
Branch with paper.md (empty if default branch): joss_paper
Version: v0.3.1
Editor: @crvernon
Reviewers: @xavieryao, @emily-sexton
Archive: Pending

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/0fbc09de7fa4e873ac68c3cb30afdd66"><img src="https://joss.theoj.org/papers/0fbc09de7fa4e873ac68c3cb30afdd66/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/0fbc09de7fa4e873ac68c3cb30afdd66/status.svg)](https://joss.theoj.org/papers/0fbc09de7fa4e873ac68c3cb30afdd66)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@xavieryao & @emily-sexton, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @crvernon know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @emily-sexton

📝 Checklist for @xavieryao

The text was updated successfully, but these errors were encountered:

editorialbot · 2024-12-04T20:56:53Z

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot · 2024-12-04T20:57:33Z

Software report:

github.com/AlDanial/cloc v 1.90  T=0.41 s (230.7 files/s, 276386.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                            11              0              0          99452
Python                          58            695           1920           3399
Jupyter Notebook                11              0           6138            834
TeX                              2             93             78            474
Markdown                         7            156              0            415
TOML                             1              5              0             57
YAML                             3              9             10             52
CSV                              2              0              0              2
-------------------------------------------------------------------------------
SUM:                            95            958           8146         104685
-------------------------------------------------------------------------------

Commit count by author:

    72	Dylan Bouchard
     6	Viren Bajaj
     5	zeya30
     4	Mohit Singh Chauhan
     2	David Skarbrevik
     2	Zeya Ahmad
     2	virenbajaj
     1	Vasistha

editorialbot · 2024-12-04T20:57:37Z

Paper file info:

📄 Wordcount for paper.md is 1582

✅ The paper includes a Statement of need section

editorialbot · 2024-12-04T20:57:38Z

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1162/tacl_a_00240 is OK
- 10.1145/3442188.3445924 is OK
- 10.18653/v1/2021.naacl-main.191 is OK
- 10.18653/v1/2022.findings-acl.165 is OK
- 10.18653/v1/2020.findings-emnlp.311 is OK
- 10.1145/3576840.3578295 is OK
- 10.1016/j.simpa.2024.100619 is OK
- 10.5281/zenodo.12608602 is OK
- 10.18653/v1/N19-3002 is OK
- 10.1145/3604915.3608860 is OK
- 10.18653/v1/2022.naacl-main.122 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Gender Bias in Coreference Resolution
- No DOI given, and none found for title: Unmasking Contextual Stereotypes: Measuring and Mi...
- No DOI given, and none found for title: TrustGPT: A Benchmark for Trustworthy and Responsi...
- No DOI given, and none found for title: Holistic Evaluation of Language Models
- No DOI given, and none found for title: DecodingTrust: A Comprehensive Assessment of Trust...
- No DOI given, and none found for title: GitHub - huggingface/evaluate: Evaluate: A library...
- No DOI given, and none found for title: Beyond the Imitation Game: Quantifying and extrapo...
- No DOI given, and none found for title: TrustLLM: Trustworthiness in Large Language Models
- No DOI given, and none found for title: AI Fairness 360:  An Extensible Toolkit for Detect...
- No DOI given, and none found for title: Fairlearn: Assessing and Improving Fairness of AI ...
- No DOI given, and none found for title: Aequitas: A Bias and Fairness Audit Toolkit
- No DOI given, and none found for title: GitHub - tensorflow/fairness-indicators: Tensorflo...
- No DOI given, and none found for title: LiFT: A Scalable Framework for Measuring Fairness ...
- No DOI given, and none found for title: An Actionable Framework for Assessing Bias and Fai...
- No DOI given, and none found for title: Towards Auditing Large Language Models: Improving ...
- No DOI given, and none found for title: Counterfactual Fairness
- No DOI given, and none found for title: Bias and Fairness in Large Language Models: A Surv...
- No DOI given, and none found for title: GitHub - huggingface/evaluate: Evaluate: A library...

❌ MISSING DOIs

- 10.18653/v1/n18-2003 may be a valid DOI for title: Gender Bias in Coreference Resolution: Evaluation ...
- 10.18653/v1/2021.findings-emnlp.211 may be a valid DOI for title: Collecting a Large-Scale Gender Bias Dataset for C...
- 10.18653/v1/2021.acl-long.416 may be a valid DOI for title: StereoSet: Measuring stereotypical bias in pretrai...
- 10.18653/v1/2020.emnlp-main.154 may be a valid DOI for title: CrowS-Pairs: A Challenge Dataset for Measuring Soc...
- 10.18653/v1/2023.acl-long.507 may be a valid DOI for title: WinoQueer: A Community-in-the-Loop Benchmark for A...
- 10.18653/v1/2021.acl-long.151 may be a valid DOI for title: RedditBias: A Real-World Resource for Bias Evaluat...
- 10.18653/v1/2022.emnlp-main.646 may be a valid DOI for title: Perturbation Augmentation for Fairer NLP
- 10.18653/v1/s18-2005 may be a valid DOI for title: Examining Gender and Race Bias in Two Hundred Sent...
- 10.18653/v1/2020.findings-emnlp.301 may be a valid DOI for title: RealToxicityPrompts: Evaluating Neural Toxic Degen...
- 10.1109/tvcg.2019.2934619 may be a valid DOI for title: The What-If Tool: Interactive Probing of Machine L...
- 10.18653/v1/2020.findings-emnlp.7 may be a valid DOI for title: Reducing Sentiment Bias in Language Models via Cou...
- 10.1145/2783258.2783311 may be a valid DOI for title: Certifying and removing disparate impact
- 10.18653/v1/2021.acl-long.150 may be a valid DOI for title: Intrinsic Bias Metrics Do Not Correlate with Appli...

❌ INVALID DOIs

- None

editorialbot · 2024-12-04T20:57:38Z

License info:

🟡 License found: Other (Check here for OSI approval)

crvernon · 2024-12-04T20:58:22Z

👋 @dylanbouchard, @xavieryao, and @emily-sexton - This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.

Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #7570 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

editorialbot · 2024-12-04T20:58:58Z

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

emily-sexton · 2024-12-06T14:14:25Z

Review checklist for @emily-sexton

Conflict of interest

I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

I confirm that I read and will adhere to the JOSS code of conduct.

General checks

Repository: Is the source code for this software available at the https://github.com/cvs-health/langfair?
License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
Contribution and authorship: Has the submitting author (@dylanbouchard) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
State of the field: Do the authors describe how this software compares to other commonly-used packages?
Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

xavieryao · 2024-12-17T00:37:25Z

Review checklist for @xavieryao

Conflict of interest

I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

I confirm that I read and will adhere to the JOSS code of conduct.

General checks

Repository: Is the source code for this software available at the https://github.com/cvs-health/langfair?
License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
Contribution and authorship: Has the submitting author (@dylanbouchard) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1. Contribute to the software 2. Report issues or problems with the software 3. Seek support

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
State of the field: Do the authors describe how this software compares to other commonly-used packages?
Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

emily-sexton · 2024-12-21T16:53:40Z

@dylanbouchard - I can see that there's a reference, 'Beyond the imitation game...' that has the author listed as 'authors, B.' When I go to the reference, it looks like there are hundreds of authors. Is this a way of referencing a paper with too many authors to list?
Also, were the missing DOIs mentioned about addressed?
@crvernon

dylanbouchard · 2024-12-21T20:51:25Z

@dylanbouchard - I can see that there's a reference, 'Beyond the imitation game...' that has the author listed as 'authors, B.' When I go to the reference, it looks like there are hundreds of authors. Is this a way of referencing a paper with too many authors to list? Also, were the missing DOIs mentioned about addressed? @crvernon

@emily-sexton, that is my understanding yes. All of the references are auto-populated using bibtex, so that would be my guess.

emily-sexton · 2024-12-22T17:28:44Z

@dylanbouchard - I'm having troubles installing langfair because I have Python 3.13.1 and looks like langfair can only be used with Python <3.12, >=3.9. I'm trying to install an old version of python in a virtual environment but the best documentation i can find on stack overflow is many years old (https://stackoverflow.com/questions/5506110/is-it-possible-to-install-another-version-of-python-to-virtualenv) and I'm struggling.

Can you give some guidance on how best to install langfair if I have a newer version of python? Should I be installing an older version of python in a virtual environment? If so, do you know the best way to do that? Thanks!!

dylanbouchard · 2024-12-22T18:06:06Z

@emily-sexton I have written a new answer to your question below. This can be done with venv as follows:

python3.9 -m venv myenv

xavieryao · 2024-12-29T23:52:23Z

@dylanbouchard great work! This sets an example of high quality (open source) software project.

Some minor issues that can be easily patched:

Minor text errors in the readme and a notebook: Fix broken links in README and copy-paste errors in example notebook cvs-health/langfair#81
Omit the brackets when referring to the author as part of a sentence: “For a case study on writers block, see Upper (1974).” Such as at line 71 of the paper. See JOSS guideline.
Paper line 65: can you add an (informal) introduction of FTU? I understand that you have formal definitions in the technical companion, but it would be more helpful for the readers.
Paper line 118: explicitly mention that the steps are what have been described previously?
Add the missing DOIs if possible.

My understanding is that the metrics for recommendation and classification are not specific to LLMs. I see that it is convenient to have them in one package, as these are also applications of LLMs. Could you add a discussion of existing repositories that offer those generic fairness metrics? Examples:
https://github.com/tensorflow/fairness-indicators
https://github.com/Trusted-AI/AIF360
https://github.com/amazon-science/generalized-fairness-metrics?tab=readme-ov-file

Is there a clearer pointer to "off-the-shelf FTU check" (line 67) other than in an example notebook? cvs-health/langfair#82

crvernon · 2024-12-30T15:25:13Z

@editorialbot generate pdf

editorialbot · 2024-12-30T15:26:43Z

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

crvernon · 2024-12-30T15:30:08Z

@editorialbot remind me in 1 week

@dylanbouchard - I can see that there's a reference, 'Beyond the imitation game...' that has the author listed as 'authors, B.' When I go to the reference, it looks like there are hundreds of authors. Is this a way of referencing a paper with too many authors to list? Also, were the missing DOIs mentioned about addressed? @crvernon

@emily-sexton, that is my understanding yes. All of the references are auto-populated using bibtex, so that would be my guess.

Let me check on this. I'll provide feedback when I make sure that we have a consistent protocol on this type of publication. Thanks!

editorialbot · 2024-12-30T15:30:10Z

Reminder set for @crvernon in 1 week

dylanbouchard · 2024-12-31T19:27:24Z

@dylanbouchard great work! This sets an example of high quality (open source) software project.

Some minor issues that can be easily patched:

Minor text errors in the readme and a notebook: Fix broken links in README and copy-paste errors in example notebook cvs-health/langfair#81

Omit the brackets when referring to the author as part of a sentence: “For a case study on writers block, see Upper (1974).” Such as at line 71 of the paper. See JOSS guideline.

Paper line 65: can you add an (informal) introduction of FTU? I understand that you have formal definitions in the technical companion, but it would be more helpful for the readers.

Paper line 118: explicitly mention that the steps are what have been described previously?

Add the missing DOIs if possible.

My understanding is that the metrics for recommendation and classification are not specific to LLMs. I see that it is convenient to have them in one package, as these are also applications of LLMs. Could you add a discussion of existing repositories that offer those generic fairness metrics? Examples: https://github.com/tensorflow/fairness-indicators https://github.com/Trusted-AI/AIF360 https://github.com/amazon-science/generalized-fairness-metrics?tab=readme-ov-file

Is there a clearer pointer to "off-the-shelf FTU check" (line 67) other than in an example notebook? cvs-health/langfair#82

Thank you @xavieryao! Really appreciate the helpful suggestions. Working on these changes now!

xavieryao · 2025-01-01T03:33:10Z

@dylanbouchard Thank you for the quick responses. All of my concerns have been adequately addressed and I think both the paper and the project are of great quality.

@crvernon I have no further comments and would recommend Accept.

Happy new year.

dylanbouchard · 2025-01-01T15:02:04Z

Thank you very much @xavieryao !

dylanbouchard · 2025-01-01T23:31:52Z

@editorialbot generate pdf

editorialbot · 2025-01-01T23:33:29Z

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

dylanbouchard · 2025-01-02T19:42:09Z

@crvernon two questions:

Should the submission be updated to the version that includes the updates recommended and approved by @xavieryao, i.e. v0.3.1?
Is there a preferred way to handle text contained in backticks from spilling over the margin? This appears to be happening in line 55 with the mention of CounterfactualGenerator

crvernon · 2025-01-03T15:32:13Z

@editorialbot set v0.3.1 as version

@dylanbouchard - yes to the version update

editorialbot · 2025-01-03T15:32:16Z

Done! version is now v0.3.1

crvernon · 2025-01-03T15:51:02Z

@dylanbouchard I also have an open request in for our development team to check out how to best represent the overflow generated by using backticks. I'll let you know when I hear back.

Concerning the @emily-sexton raised about the many author paper and how to cite it:

This is the correct reference format to add to your bib file:

@Article{srivastava2022beyond,
title={Beyond the imitation game: Quantifying and extrapolating the capabilities of language models},
author={Srivastava, Aarohi and Rastogi, Abhinav and Rao, Abhishek and Shoeb, Abu Awal Md and Abid, Abubakar and Fisch, Adam and Brown, Adam R and Santoro, Adam and Gupta, Aditya and Garriga-Alonso, Adri{`a} and others},
journal={arXiv preprint arXiv:2206.04615},
year={2022},
doi={10.48550/arXiv.2206.04615}
}

Also, for your references containing no date: please either cite a release of these repositories or simply use the weblink in text. The citation is preferred. Thanks!

dylanbouchard · 2025-01-03T17:37:06Z

@editorialbot generate pdf

dylanbouchard · 2025-01-03T17:37:51Z

@dylanbouchard I also have an open request in for our development team to check out how to best represent the overflow generated by using backticks. I'll let you know when I hear back.

Concerning the @emily-sexton raised about the many author paper and how to cite it:

This is the correct reference format to add to your bib file:

@Article{srivastava2022beyond,
title={Beyond the imitation game: Quantifying and extrapolating the capabilities of language models},
author={Srivastava, Aarohi and Rastogi, Abhinav and Rao, Abhishek and Shoeb, Abu Awal Md and Abid, Abubakar and Fisch, Adam and Brown, Adam R and Santoro, Adam and Gupta, Aditya and Garriga-Alonso, Adri{`a} and others},
journal={arXiv preprint arXiv:2206.04615},
year={2022},
doi={10.48550/arXiv.2206.04615}
}

Also, for your references containing no date: please either cite a release of these repositories or simply use the weblink in text. The citation is preferred. Thanks!

Thank you @crvernon! The requested changes have been made to the references.

editorialbot · 2025-01-03T17:38:42Z

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

dylanbouchard · 2025-01-04T16:54:19Z

@dylanbouchard - I'm having troubles installing langfair because I have Python 3.13.1 and looks like langfair can only be used with Python <3.12, >=3.9. I'm trying to install an old version of python in a virtual environment but the best documentation i can find on stack overflow is many years old (https://stackoverflow.com/questions/5506110/is-it-possible-to-install-another-version-of-python-to-virtualenv) and I'm struggling.

Can you give some guidance on how best to install langfair if I have a newer version of python? Should I be installing an older version of python in a virtual environment? If so, do you know the best way to do that? Thanks!!

@emily-sexton I looked into this more and it I was able to specify the Python version with venv (e.g. 3.9) as follows:

python3.9 -m venv myenv

Please let me know if you have further questions. Thank you!

emily-sexton · 2025-01-04T20:44:32Z

@dylanbouchard - I'm having troubles installing langfair because I have Python 3.13.1 and looks like langfair can only be used with Python <3.12, >=3.9. I'm trying to install an old version of python in a virtual environment but the best documentation i can find on stack overflow is many years old (https://stackoverflow.com/questions/5506110/is-it-possible-to-install-another-version-of-python-to-virtualenv) and I'm struggling.
Can you give some guidance on how best to install langfair if I have a newer version of python? Should I be installing an older version of python in a virtual environment? If so, do you know the best way to do that? Thanks!!

@emily-sexton I looked into this more and it I was able to specify the Python version with venv (e.g. 3.9) as follows:

python3.9 -m venv myenv

Please let me know if you have further questions. Thank you!

@dylanbouchard - Thank you very much! I was able to create a virtual environment with python 3.9 and install langfair properly.

emily-sexton · 2025-01-04T20:59:05Z

@dylanbouchard - I just finished reviewing and think your work here is great! I have two points to bring up:

I'm getting errors when I import StereotypeMetrics, although it still seems to be running properly. I've created a bug for that here: Errors when importing StereotypeMetrics cvs-health/langfair#93
When I use the code provided in the usage examples, anytime I run one of the lines with 'await', I get an error that the await keyword is outside of an async function. Putting the code inside an async function works better for me. Do you think this should be added to the documentation or is there a better way that you get the lines with 'await' to run?

Everything else looks great and once those two things get sorted, we should be good to go!

dylanbouchard · 2025-01-04T21:40:36Z

Thank you very much @emily-sexton! I have responded to your issue with a likely solution. I believe including the below in your .py file should fix both issues you state above:

def main():
    # Your code here

if __name__ == '__main__':
    main()

editorialbot added review Track: 5 (DSAIS) Data Science, Artificial Intelligence, and Machine Learning labels Dec 4, 2024

editorialbot assigned crvernon Dec 4, 2024

editorialbot mentioned this issue Dec 4, 2024

[PRE REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7558

Closed

This was referenced Dec 31, 2024

Joss paper draft cvs-health/langfair#84

Merged

New FTU check method cvs-health/langfair#85

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7570

[REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7570

editorialbot commented Dec 4, 2024 •

edited

Loading

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

crvernon commented Dec 4, 2024

editorialbot commented Dec 4, 2024

emily-sexton commented Dec 6, 2024 •

edited

Loading

xavieryao commented Dec 17, 2024 •

edited

Loading

emily-sexton commented Dec 21, 2024

dylanbouchard commented Dec 21, 2024

emily-sexton commented Dec 22, 2024

dylanbouchard commented Dec 22, 2024 •

edited

Loading

xavieryao commented Dec 29, 2024 •

edited

Loading

crvernon commented Dec 30, 2024

editorialbot commented Dec 30, 2024

crvernon commented Dec 30, 2024

editorialbot commented Dec 30, 2024

dylanbouchard commented Dec 31, 2024

xavieryao commented Jan 1, 2025

dylanbouchard commented Jan 1, 2025 •

edited

Loading

dylanbouchard commented Jan 1, 2025

editorialbot commented Jan 1, 2025

dylanbouchard commented Jan 2, 2025

crvernon commented Jan 3, 2025

editorialbot commented Jan 3, 2025

crvernon commented Jan 3, 2025

dylanbouchard commented Jan 3, 2025

dylanbouchard commented Jan 3, 2025

editorialbot commented Jan 3, 2025

dylanbouchard commented Jan 4, 2025

emily-sexton commented Jan 4, 2025

emily-sexton commented Jan 4, 2025

dylanbouchard commented Jan 4, 2025

[REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7570

[REVIEW]: LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases #7570

Comments

editorialbot commented Dec 4, 2024 • edited Loading

Status

Reviewer instructions & questions

Checklists

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

editorialbot commented Dec 4, 2024

crvernon commented Dec 4, 2024

editorialbot commented Dec 4, 2024

emily-sexton commented Dec 6, 2024 • edited Loading

Review checklist for @emily-sexton

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

xavieryao commented Dec 17, 2024 • edited Loading

Review checklist for @xavieryao

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

emily-sexton commented Dec 21, 2024

dylanbouchard commented Dec 21, 2024

emily-sexton commented Dec 22, 2024

dylanbouchard commented Dec 22, 2024 • edited Loading

xavieryao commented Dec 29, 2024 • edited Loading

crvernon commented Dec 30, 2024

editorialbot commented Dec 30, 2024

crvernon commented Dec 30, 2024

editorialbot commented Dec 30, 2024

dylanbouchard commented Dec 31, 2024

xavieryao commented Jan 1, 2025

dylanbouchard commented Jan 1, 2025 • edited Loading

dylanbouchard commented Jan 1, 2025

editorialbot commented Jan 1, 2025

dylanbouchard commented Jan 2, 2025

crvernon commented Jan 3, 2025

editorialbot commented Jan 3, 2025

crvernon commented Jan 3, 2025

dylanbouchard commented Jan 3, 2025

dylanbouchard commented Jan 3, 2025

editorialbot commented Jan 3, 2025

dylanbouchard commented Jan 4, 2025

emily-sexton commented Jan 4, 2025

emily-sexton commented Jan 4, 2025

dylanbouchard commented Jan 4, 2025

editorialbot commented Dec 4, 2024 •

edited

Loading

emily-sexton commented Dec 6, 2024 •

edited

Loading

xavieryao commented Dec 17, 2024 •

edited

Loading

dylanbouchard commented Dec 22, 2024 •

edited

Loading

xavieryao commented Dec 29, 2024 •

edited

Loading

dylanbouchard commented Jan 1, 2025 •

edited

Loading