Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GPU installation on Windows: compilation of thinc_gpu_ops #2855

Closed
pythonBerg opened this issue Oct 16, 2018 · 19 comments
Closed

GPU installation on Windows: compilation of thinc_gpu_ops #2855

pythonBerg opened this issue Oct 16, 2018 · 19 comments
Labels
gpu Using spaCy on GPU install Installation issues 🔮 thinc spaCy's machine learning library Thinc

Comments

@pythonBerg
Copy link

New clean install of spacy[cuda9.2]. Seeing several issues.

  1. CYMEM warning submitted in ValueError: cymem.cymem.Pool has the wrong size, try recompiling. #2852 generated in all operations.

  2. When using ner training on new model...get error that thinc_gpu_ops has no function "hash".

  3. In classifier, error is that "Only Cupy Arrays can be concatenated"

This script has worked without gpu for me for months (in ubuntu 18.04) and is slightly modified version of one posted in spacy examples

********************************ner training error ***************
Created blank 'en' model
Loading NER Training Data
Traceback (most recent call last):
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\pdb.py", line 1667, in main
pdb._runscript(mainpyfile)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\pdb.py", line 1548, in _runscript
self.run(statement)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\bdb.py", line 434, in run
exec(cmd, globals, locals)
File "", line 1, in
File "c:\home\modeltrainer.py", line 10, in
Compatible with: spaCy v2.0.0+"""
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\plac_core.py", line 328, in call
cmd, result = parser.consume(arglist)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\plac_core.py", line 207, in consume
return cmd, self.func((args + varargs + extraopts), **kwargs)
File "c:\home\modeltrainer.py", line 173, in main
losses=losses)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\spacy\language.py", line 421, in update
proc.update(docs, golds, drop=drop, sgd=get_grads, losses=losses)
File "nn_parser.pyx", line 565, in spacy.syntax.nn_parser.Parser.update
File "nn_parser.pyx", line 729, in spacy.syntax.nn_parser.Parser.get_batch_model
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 61, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 280, in begin_update
drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 61, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 374, in uniqued_fwd
Y_uniq, bp_Y_uniq = layer.begin_update(X_uniq, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 61, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 258, in wrap
output = func(*args, **kwargs)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 258, in wrap
output = func(*args, **kwargs)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 258, in wrap
output = func(*args, **kwargs)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\neural_classes\hash_embed.py", line 51, in begin_update
keys = self.ops.hash(ids, self.seed) % self.nV
File "ops.pyx", line 821, in thinc.neural.ops.CupyOps.hash
AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'
***************************************classifier error *****************

File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\plac_core.py", line 328, in call
cmd, result = parser.consume(arglist)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\plac_core.py", line 207, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "c:\home\modeltrainer.py", line 128, in main
losses=losses)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\spacy\language.py", line 421, in update
proc.update(docs, golds, drop=drop, sgd=get_grads, losses=losses)
File "pipeline.pyx", line 876, in spacy.pipeline.TextCategorizer.update
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 61, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 176, in
values = [fwd(X, *a, **k) for fwd in forward]
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 258, in wrap
output = func(*args, **kwargs)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\thinc\api.py", line 61, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\spacy_ml.py", line 102, in _preprocess_doc
keys = ops.xp.concatenate(keys)
File "c:\Users\MPC-LAPTOP01\AppData\Local\Programs\Python\Python36\lib\site-packages\cupy\manipulation\join.py", line 49, in concatenate
return core.concatenate_method(tup, axis)
File "cupy\core\core.pyx", line 2728, in cupy.core.core.concatenate_method
File "cupy\core\core.pyx", line 2741, in cupy.core.core.concatenate_method
TypeError: Only cupy arrays can be concatenated
****env ****

Windows10
Python 3.6
spacy 2.0.16
thinc 6.12.0
thinc-gpu-ops 0.0.3

script*********
from future import unicode_literals, print_function
import plac
import random
import sys
import os
from pathlib import Path
import thinc.extra.datasets
import psycopg2
import spacy
from spacy.util import minibatch, compounding
import re
import regex
from collections import defaultdict
spacy.prefer_gpu()

@plac.annotations(
model=("Model name. Defaults to blank 'en' model.", "option", "m", str),
output_dir=("Optional output directory", "option", "o", Path),
n_texts=("Number of texts to train from", "option", "t", int),
datatype=("Type of object to train", "option", "d", str),
n_iter=("Number of training iterations", "option", "n", int))

def main(model=None, output_dir='/home/classify/section999', n_iter=30, n_texts=4000,datatype='all'):
if not model and output_dir:
model=output_dir
if datatype is None:
exit(1)
else:
try:
nlp = spacy.load(model) # load existing spaCy model
print("Loaded model '%s'" % model)
except:
nlp = spacy.blank('en') # create blank Language class
print("Created blank 'en' model")

if datatype == 'docparts' or datatype == 'all':
    usecat='docpart'
    rc = cur.execute("select classification from class_map where group_name = 'docpart'")
    labeldata = cur.fetchall()
    lst = [item[0] for item in labeldata]
    
    if 'textcat' not in nlp.pipe_names:
        textcat = nlp.create_pipe('textcat')
        nlp.add_pipe(textcat, last=True)
    else:
        textcat = nlp.get_pipe('textcat')

    for lab in lst:
        textcat.add_label(lab)

    limit=0
    split=.80

    train_data=[]
    traincats=[]
    rc = cur.execute("select classification, text from document_sections where classification is not null and classification != '' and classification != 'Not Classified' and reviewed=1")
    data1 = cur.fetchall()
    for row in data1:
        txt = row[1]
        txt = re.sub('<tbody>|</tbody>|<table.*?>|<tr.*?>|</table>|<td.*?>|<th.*?>','',txt)
        txt = re.sub('</tr>|<br>','\n',txt)
        txt = re.sub('</th>|</td>','\t',txt)
        txt = re.sub('\self+',' ',txt)
        txt = re.sub('\n','',txt)
        txt = txt.strip()
        traincats.append(row[0])
        train_data.append([txt,row[0]])
  
    
    useCats=set(traincats)
    random.shuffle(train_data)
    train_data = train_data[-limit:]
    texts, labels = zip(*train_data)
    cats=[]
    for lab in labels:
        cd = {}
        for cat in useCats:
            cd[cat] = bool(cat==lab)
        cats.append(cd)

    split = int(len(train_data) * split)
    train_texts=texts[:split]
    train_cats = cats[:split]
    dev_texts=texts[split:]
    dev_cats=cats[split:]
    for lab in useCats:
        textcat.add_label(lab)

    print("Using {} examples ({} training, {} evaluation)"
          .format(n_texts, len(train_texts), len(dev_texts)))
    train_data = list(zip(train_texts,
                          [{'cats': cats} for cats in train_cats]))

    other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'textcat']
    with nlp.disable_pipes(*other_pipes):  # only train textcat
        nlp.vocab.vectors.name = 'spacy_pretrained_vectors'
        optimizer = nlp.begin_training()
        print("Training the model...")
        print('{:^5}\t{:^5}\t{:^5}\t{:^5}'.format('LOSS', 'P', 'R', 'F'))
        for i in range(n_iter):
            losses = {}
            # batch up the examples using spaCy's minibatch
            batches = minibatch(train_data, size=compounding(4., 32., 1.001))
            for batch in batches:
                texts, annotations = zip(*batch)
                nlp.update(texts, annotations, sgd=optimizer, drop=0.2,
                           losses=losses)
            with textcat.model.use_params(optimizer.averages):
                scores = evaluate(nlp.tokenizer, textcat, dev_texts, dev_cats)
            print('{0:.3f}\t{1:.3f}\t{2:.3f}\t{3:.3f}'  # print a simple table
                  .format(losses['textcat'], scores['textcat_p'],
                              scores['textcat_r'], scores['textcat_f']))

    if output_dir is not None:
        output_dir = Path(output_dir)
        if not output_dir.exists():
            output_dir.mkdir()
        nlp.to_disk(output_dir)
        print("Saved model to", output_dir)

if datatype == 'ner' or datatype == 'all':

    if 'ner' not in nlp.pipe_names:
        ner = nlp.create_pipe('ner')
        nlp.add_pipe(ner, last=True)
    else:
        ner = nlp.get_pipe('ner')

    print("Loading NER Training Data")
    [lst,TRAIN_DATA] = load_ner_data()

    for lab in lst:
        ner.add_label(lab)

    other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner']
    with nlp.disable_pipes(*other_pipes):  # only train NER
        nlp.vocab.vectors.name = 'spacy_pretrained_vectors'
        optimizer = nlp.begin_training()
        for itn in range(n_iter+30):
            random.shuffle(TRAIN_DATA)
            losses = {}
            for text, annotations in TRAIN_DATA:
                nlp.update(
                    [text],  # batch of texts
                    [annotations],  # batch of annotations
                    drop=0.5,  # dropout - make it harder to memorise data
                    sgd=optimizer,  # callable to update weights
                    losses=losses)
            print(itn,losses)
            if output_dir is not None and itn%5 == 0:
                output_dir = Path(output_dir)
                if not output_dir.exists():
                    output_dir.mkdir()
                nlp.to_disk(output_dir)
                print("Saved model # TODO: o", output_dir)

    if output_dir is not None:
        output_dir = Path(output_dir)
        if not output_dir.exists():
            output_dir.mkdir()
        nlp.to_disk(output_dir)
        print("Saved model # TODO: o", output_dir)

def load_ner_data():
train_data=[]
annMaster = defaultdict(list)
rc = cur.execute("select sub_document_id,model_field,token_start, token_end from new_annotation where status not in ('Deleted','Pending') and source not in ('section_classification','sub_classification') and current_version=1")
anndata = cur.fetchall()

for ann in anndata:
    annMaster[ann[0]].append((ann[2],ann[3],ann[1]))


cats=[]
for subId in annMaster.keys():
    rc = cur.execute("select text from sub_document where sub_document_id = %s",(subId,))
    data1 = cur.fetchall()
    subAnns = annMaster[subId]
    txt = data1[0][0]
    # txt = sub.text
    if not txt:
        continue
    if not subAnns:
        continue
    td = {}
    td['entities']=[]
    i=0
    for ann in subAnns:
        cats.append(ann[2])
        td['entities'].append(ann)
    ts = [txt, td]
    train_data.append(ts)
cats = set(cats)
return [cats,train_data]

def evaluate(tokenizer, textcat, texts, cats):
docs = (tokenizer(text) for text in texts)
tp = 1e-8 # True positives
fp = 1e-8 # False positives
fn = 1e-8 # False negatives
tn = 1e-8 # True negatives
for i, doc in enumerate(textcat.pipe(docs)):
gold = cats[i]
for label, score in doc.cats.items():
if label not in gold:
continue
if score >= 0.5 and gold[label] >= 0.5:
tp += 1.
elif score >= 0.5 and gold[label] < 0.5:
fp += 1.
elif score < 0.5 and gold[label] < 0.5:
tn += 1
elif score < 0.5 and gold[label] >= 0.5:
fn += 1
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f_score = 2 * (precision * recall) / (precision + recall)
return {'textcat_p': precision, 'textcat_r': recall, 'textcat_f': f_score}

if name == 'main':
plac.call(main)
@honnibal
Copy link
Member

A few things to check:

  • The correct spelling is spacy[cuda92], not spacy[cuda9.2] as you wrote. Did you typo this in your issue, or did you install the wrong package?
  • Have you set your LD_LIBRARY_PATH to include your cuda installation? Default location will be /usr/local/cuda/lib64. You might also need to set your PATH so that /usr/local/cuda/bin is on it.
  • I believe there's another issue open about text classifier training on GPU. You should continue using the CPU to train the text classifier for now.

@pythonBerg
Copy link
Author

pythonBerg commented Oct 16, 2018 via email

@honnibal honnibal added the bug Bugs and behaviour differing from documentation label Oct 17, 2018
@honnibal
Copy link
Member

Okay, thanks for the report. Wasn't able to test GPU on Windows unfortunately --- will investigate.

@Ru7z
Copy link

Ru7z commented Oct 18, 2018 •
edited
Loading

Hi, I have the same bug. Following is the details.

Environments:

  • Platform: Ubuntu 16.04.4 LTS
  • Cuda compilation tools, release 9.1, V9.1.85
  • Python version: 3.5.2
  • spaCy version: 2.0.16
  • Models: en
import spacy
spacy.require_gpu() # True
nlp = spacy.load('en_core_web_sm')
text = 'spaCy is good.'
nlp(text) # AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'

@veonua
Copy link

veonua commented Oct 23, 2018

same bug here, the root of the bug is

from thinc_gpu_ops.gpu_ops import *
print (thinc_gpu_ops.__file__)
thinc_gpu_ops.hash(1,2,3)

/anaconda/envs/py36/lib/python3.6/site-packages/thinc_gpu_ops/gpu_ops.cpython-36m-x86_64-linux-gnu.so: undefined symbol: __cudaPopCallConfiguration

installed by spacy[cuda] to Azure Deeplearning VM

$LD_LIBRARY_PATH =
/opt/intel/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64_lin/gcc4.7:/opt/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64_lin::/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64/:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64/

$PATH
/anaconda/envs/py36/bin:/home/agradmin/bin:/home/agradmin/.local/bin:/anaconda/envs/py35/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/caffe/build/install/bin/:/usr/local/cuda/bin:/dsvm/tools/cntk/cntk/bin:/usr/local/cuda/bin:/dsvm/tools/cntk/cntk/bin:/dsvm/tools/spark/current/bin:/opt/mssql-tools/bin

cuda92,

@veonua
Copy link

veonua commented Oct 26, 2018 •
edited
Loading

@honnibal how do I debug thinc_gpu_ops so

I have it working in conda py3.5, but have __cudaPopCallConfiguration on py3.6 (same machine)
Unfortunately I don't see any difference in CUDA related env. Could you please tell where to start?

@ivan-bilan
Copy link

ivan-bilan commented Dec 6, 2018 •
edited
Loading

Hi, I have the same bug. Following is the details.

Environments:

* Platform: `Ubuntu 16.04.4 LTS`

* Cuda compilation tools, release `9.1`, V9.1.85

* Python version: `3.5.2`

* spaCy version:          `2.0.16`

* Models: en

import spacy
spacy.require_gpu() # True
nlp = spacy.load('en_core_web_sm')
text = 'spaCy is good.'
nlp(text) # AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'

I have exactly the same issue but on Ubuntu 16.04. Cuda 9.2 is installed correctly, all the path variables are set. Cupy works separately too. This is the only error I get. I am on Python 3.7, however.

@ines ines added the gpu Using spaCy on GPU label Dec 6, 2018
@honnibal
Copy link
Member

Could you try again with the latest spacy-nightly? Thanks.

@honnibal honnibal added install Installation issues more-info-needed This issue needs more information and removed bug Bugs and behaviour differing from documentation labels Dec 29, 2018
@pythonBerg
Copy link
Author

pythonBerg commented Dec 29, 2018 via email

@no-response no-response bot removed the more-info-needed This issue needs more information label Dec 29, 2018
@honnibal honnibal changed the title Issues with both classifier and ner training with latest release GPU installation on Windows: compilation of thinc_gpu_ops Feb 25, 2019
@ines ines added the 🔮 thinc spaCy's machine learning library Thinc label Feb 25, 2019
@zinark
Copy link

zinark commented Mar 21, 2019

Same output:

OS : Windows 10
pip install cupy-cuda92
pip install spacy[cuda92]
pip install thinc_gpu_ops thinc[cuda92]

import spacy
spacy.require_gpu() # True
nlp = spacy.load('en_core_web_sm')
text = 'spaCy is good.'
nlp(text) # AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'

@ZeroCool2u
Copy link

ZeroCool2u commented Mar 28, 2019 •
edited
Loading

I have the exact same issue. This is on a brand new machine.
EDIT: We've also replicated this issue on other Windows 10 machines here.
The issue does not present on RHEL 7 with CUDA v9.0.

OS: Windows 10 v1709
CUDA v10.0
2080 Ti
The following commands replicate the issue.

pip install spacy[cuda100]
spacy download en_core_web_lg

python
import spacy
spacy.require_gpu()
#True
nlp = spacy.load('en_core_web_lg')
text = 'SpaCy is great.'
doc = nlp(text)

Traceback (most recent call last):
File "", line 1, in
File "C:\DevTools\Python3\lib\site-packages\spacy\language.py", line 385, in call
doc = proc(doc, **component_cfg.get(name, {}))
File "pipes.pyx", line 374, in spacy.pipeline.pipes.Tagger.call
File "pipes.pyx", line 393, in spacy.pipeline.pipes.Tagger.predict
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\model.py", line 165, in call
return self.predict(x)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\feed_forward.py", line 40, in predict
X = layer(X)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\model.py", line 165, in call
return self.predict(x)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 279, in predict
X = layer(layer.ops.flatten(seqs_in, pad=pad))
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\model.py", line 165, in call
return self.predict(x)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\feed_forward.py", line 40, in predict
X = layer(X)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\model.py", line 165, in call
return self.predict(x)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\model.py", line 129, in predict
y, _ = self.begin_update(X, drop=None)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 348, in uniqued_fwd
Y_uniq, bp_Y_uniq = layer.begin_update(X_uniq, drop=drop)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\feed_forward.py", line 46, in begin_update
X, inc_layer_grad = layer.begin_update(X, drop=drop)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 225, in wrap
output = func(*args, **kwargs)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 225, in wrap
output = func(*args, **kwargs)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 225, in wrap
output = func(*args, **kwargs)
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in begin_update
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 132, in
values = [fwd(X, *a, **k) for fwd in forward]
File "C:\DevTools\Python3\lib\site-packages\thinc\api.py", line 225, in wrap
output = func(*args, **kwargs)
File "C:\DevTools\Python3\lib\site-packages\thinc\neural_classes\hash_embed.py", line 55, in begin_update
keys = self.ops.hash(ids, self.seed) % self.nV
File "ops.pyx", line 917, in thinc.neural.ops.CupyOps.hash
AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'

@alexookah
Copy link

i have the same issue on windows 10
you should also change the cuda version 101 to support it since cupy-cuda101 is available

@bklippstein
Copy link

i have the same bug on Mint 19:
module 'thinc_gpu_ops' has no attribute 'hash'

@jfelectron
Copy link

For Linux (Ubuntu), I had this problem because of different gcc versions (newer version of the nvidia driver and cuda will install gcc-8, but cupy and others dependices use nvcc, which only works with gcc7 or lower), When running builds, make sure CUDA_HOME is set and CUDA_HOME/bin is on your PATH.

@regrettablemouse136
Copy link

I have the same issue.

Env:
OS = Windows 10.
Cuda = 9
SpaCy = 2.1.7
Python = 3.6.8
thinc_gpu_ops = 0.0.4

Error:
ops.pyx in thinc.neural.ops.CupyOps.hash()

AttributeError: module 'thinc_gpu_ops' has no attribute 'hash'

@mrxiaohe
Copy link

mrxiaohe commented Aug 9, 2019

Is there any additional update on thinc_gpu_ops? I have not been able to correctly compile it for the past year, on Windows, specifically.

@pythonBerg
Copy link
Author

This is same as #3554. Lot's of people have hit a wall. Many have spent significant time trying to resolve with no published success

@pythonBerg
Copy link
Author

See explosion/thinc#92 (comment) for a solution that has worked for me. Thanks to mtmiron

@lock
Copy link

lock bot commented Oct 1, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Oct 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
gpu Using spaCy on GPU install Installation issues 🔮 thinc spaCy's machine learning library Thinc
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests