Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] split with regular expression support #4658

Closed
viadea opened this issue Jan 28, 2022 · 2 comments
Closed

[FEA] split with regular expression support #4658

viadea opened this issue Jan 28, 2022 · 2 comments
Labels
duplicate This issue or pull request already exists feature request New feature or request

Comments

@viadea
Copy link
Collaborator

viadea commented Jan 28, 2022

I wish we support split with regular expressions.

Mini repro:

val address = Seq((1,"abc.com"),
(2,"...abc"),
(3,".a.b.c"))

import spark.implicits._
val df = address.toDF("id","txt")
df.write.mode("overwrite").format("parquet").save("/tmp/testparquet")
val df2=spark.read.parquet("/tmp/testparquet")
df2.createOrReplaceTempView("df2")
spark.sql("select id,txt, split(txt, '\\\\.')[0] AS new_txt from df2").show()

Not-supported-messages:

!Expression <StringSplit> split(txt#209, \., -1) cannot run on GPU because regular expressions are not supported yet
@viadea viadea added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jan 28, 2022
@sameerz
Copy link
Collaborator

sameerz commented Jan 28, 2022

Same as issue #4003

@sameerz sameerz added the duplicate This issue or pull request already exists label Jan 28, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 1, 2022
@sameerz
Copy link
Collaborator

sameerz commented Feb 1, 2022

Closing as duplicate

@sameerz sameerz closed this as completed Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants