Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Go][Parquet] Support fallback encoding #31

Open
shaoting-huang opened this issue Jul 16, 2024 · 0 comments
Open

[Go][Parquet] Support fallback encoding #31

shaoting-huang opened this issue Jul 16, 2024 · 0 comments
Labels

Comments

@shaoting-huang
Copy link

Describe the enhancement requested

WithEncoding comments are misleading users now.

// WithEncoding defines the encoding that is used when we aren't using dictionary encoding.
//
// This is either applied if dictionary encoding is disabled, or if we fallback if the dictionary
// grew too large.
func WithEncoding(encoding Encoding) WriterProperty {
	return func(cfg *writerPropConfig) {
		if encoding == Encodings.PlainDict || encoding == Encodings.RLEDict {
			panic("parquet: can't use dictionary encoding as fallback encoding")
		}
		cfg.wr.defColumnProps.Encoding = encoding
	}
}

Currently, Go parquet fallback encoding is hardcoded as plain encoding. See fallbackToPlain. It is better to support custom fallback encoding.

Component(s)

Go, Parquet

@shaoting-huang shaoting-huang added the Type: enhancement New feature or request label Jul 16, 2024
@assignUser assignUser transferred this issue from apache/arrow Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant