Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osx build with unittest #12

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
e30e71f
osx build with unittest
gyanesh-m Feb 28, 2018
3e19aa0
minor fix
gyanesh-m Feb 28, 2018
66872b2
Add travis file
gyanesh-m Feb 28, 2018
ea24365
Merge branch 'osx-build-test' of https://github.com/gyanesh-m/gensim-…
gyanesh-m Feb 28, 2018
6d73acf
Make unittest verbose and increase timeout
gyanesh-m Feb 28, 2018
7c58269
Add fast version check, make doc2vec tests verbose & revert travis fi…
gyanesh-m Mar 1, 2018
a86f216
Add tests in forge_test.sh
gyanesh-m Mar 1, 2018
17f2041
fix test_fast check
gyanesh-m Mar 1, 2018
4648966
Fetch test_fast.py and then try
gyanesh-m Mar 1, 2018
a0d1e01
Retry with curl
gyanesh-m Mar 2, 2018
107faad
Make debug logs visible.
gyanesh-m Mar 3, 2018
ace8fd1
Fetch and debug doc2vec test
gyanesh-m Mar 3, 2018
5a284bd
Display fast version
gyanesh-m Mar 5, 2018
97eb1aa
Display fast version
gyanesh-m Mar 5, 2018
21bb7c0
Debug tests for word2vec and fasttext
gyanesh-m Mar 5, 2018
9ba5e5e
Try fasttext first
gyanesh-m Mar 6, 2018
65bac25
Merge branch 'osx-build-test' of https://github.com/gyanesh-m/gensim-…
gyanesh-m Mar 11, 2018
770a8de
limit workaround
gyanesh-m Mar 11, 2018
2593aa8
Retry after curl
gyanesh-m Mar 11, 2018
2d40131
Remove accuracy test and try rest
gyanesh-m Mar 12, 2018
2efbe68
Test for complete w2v
gyanesh-m Mar 12, 2018
980ef3d
Add profiling for word2vec
gyanesh-m Mar 28, 2018
4b36ce3
Fix minor error
gyanesh-m Mar 28, 2018
1ccf055
Retry with pytest
gyanesh-m Mar 29, 2018
565ea05
Fix errors and retry using existing test code
gyanesh-m Mar 30, 2018
7468d9a
Fix minor error
gyanesh-m Mar 30, 2018
f651347
Fix minor error
gyanesh-m Mar 30, 2018
f74bc62
Add dependency for meta.yaml
gyanesh-m Mar 31, 2018
8c382de
Install pytest-profiling from source.
gyanesh-m Mar 31, 2018
62156e2
Change profiler to cprofile and retry
gyanesh-m Apr 18, 2018
1e72ae1
Add blas and openblas requirement
gyanesh-m Apr 20, 2018
55ce812
Remove variant
gyanesh-m Apr 20, 2018
b1d757a
Set number of threads to 1 and retry
gyanesh-m Apr 20, 2018
b7de836
Add path information
gyanesh-m Apr 22, 2018
cdbac2d
minor fix
gyanesh-m Apr 22, 2018
4e1b59a
Add flags info
gyanesh-m Apr 22, 2018
b9c9b7c
Remove thread restriction
gyanesh-m Apr 22, 2018
cf68967
Add path info
gyanesh-m Apr 23, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# This file was generated automatically from conda-smithy. To update this configuration,
# update the conda-forge.yml and/or the recipe/meta.yaml.

language: generic

os: osx
osx_image: xcode6.4

env:
matrix:

- CONDA_PY=27
- CONDA_PY=35
- CONDA_PY=36
global:
# The BINSTAR_TOKEN secure variable. This is defined canonically in conda-forge.yml.
- secure: "ecN7i24QmsWbGKfWOmzAj56rvl5Hh298I3OOFeED99DxJKlpucgxgXMOC2rYDVU8bMRVGZZxLkN1sMyowyM9uaWzrL/N6pA27AvOImMWGMVy7MOYhWHsTK73NAhuIxMMByqtQiOs9gEiOpJaQwxOQSDynae4rSCvul1bTeOyOcbsdk2cT8mkEKfUxy6M50wA20TvW/Q88lEmnC0C1Pa0dysJ85G29jBuH8yh3p4KwtiEyBNfvffqeQuUo1qJWNr9YEXXi/Aq18+DDsR3vLcSHSXMuKUHQ/TE7gq2LAAWEYkEGtOGiQFNeoFujSdmZ5dom6N9uLo/0Ta75VCZoytVgJTADgeghinyBCTBgoXjy3C4e7k/DHaBGdS1ZD42s4MLOYeAU8nJbjTGIlMzdfLzO6KjkLGTk4TBQm7/sDJiSUlf5wq9SrTB3O1UIlwUM5sS5MUC+xg3Btmz/W0HvtzI66oQoFjoh3QQnbGJM1MqpYqcR8QJRofSNfFzW8WDPhEm3bn8AlbjyNY1v9C0zZgHTRuT1Put46u7T0O2W4p6/0U+dsfv+6Iq5VxeKO2ptZRRpBX95Kf3KwEFgHyFsk6BQArU9Ijo3Ud+39wVcD5y9P9iKVugvdoBtL3yl74royeaXZXVwmhKYTYRTBMxe4Ab1Oqdan469Mj5d2JmTnYiKF4="


before_install:
# Fast finish the PR.
- |
(curl https://raw.githubusercontent.com/conda-forge/conda-forge-build-setup-feedstock/master/recipe/ff_ci_pr_build.py | \
python - -v --ci "travis" "${TRAVIS_REPO_SLUG}" "${TRAVIS_BUILD_NUMBER}" "${TRAVIS_PULL_REQUEST}") || exit 1

# Remove homebrew.
- |
echo ""
echo "Removing homebrew from Travis CI to avoid conflicts."
curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall > ~/uninstall_homebrew
chmod +x ~/uninstall_homebrew
~/uninstall_homebrew -fq
rm ~/uninstall_homebrew


install:
# Install Miniconda.
- |
echo ""
echo "Installing a fresh version of Miniconda."
MINICONDA_URL="https://repo.continuum.io/miniconda"
MINICONDA_FILE="Miniconda3-latest-MacOSX-x86_64.sh"
curl -L -O "${MINICONDA_URL}/${MINICONDA_FILE}"
bash $MINICONDA_FILE -b

# Configure conda.
- |
echo ""
echo "Configuring conda."
source /Users/travis/miniconda3/bin/activate root
conda config --remove channels defaults
conda config --add channels defaults
conda config --add channels conda-forge
conda config --set show_channel_urls true
conda install --yes --quiet conda-forge-build-setup
source run_conda_forge_build_setup

script:
- curl https://raw.githubusercontent.com/gyanesh-m/gensim-feedstock/770a8defdafd57c11a719a5206b514204367c8cc/recipe/limit_breaker.sh > limit_breaker.sh
- bash limit_breaker.sh

- upload_or_check_non_existence ./recipe conda-forge --channel=main
18 changes: 17 additions & 1 deletion recipe/forge_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,22 @@ set -e -x
unset REQUESTS_CA_BUNDLE
unset SSL_CERT_FILE

export LIBRARY_PATH="${PREFIX}/lib"
export C_INCLUDE_PATH="${PREFIX}/include"
export CPLUS_INCLUDE_PATH="${PREFIX}/include"

if [[ $(uname) == 'Darwin' ]]; then
export CFLAGS="${CFLAGS} -stdlib=libc++ -lc++"
export LDFLAGS="-headerpad_max_install_names -undefined dynamic_lookup -bundle -Wl,-search_paths_first -lc++"
else
unset LDFLAGS
fi

python -m pip install -U pip nose
pip install -U gensim
nosetests --exe -v gensim
curl https://raw.githubusercontent.com/gyanesh-m/gensim-feedstock/osx-build-test/recipe/test_fast.py>test_fast.py
python test_fast.py
curl https://raw.githubusercontent.com/gyanesh-m/gensim-feedstock/osx-build-test/recipe/profiler.py>profiler.py
python profiler.py
tar -zcvf prof.tar.gz ./combined-*
curl --upload-file prof.tar.gz https://transfer.sh/prof.tar.gz
39 changes: 39 additions & 0 deletions recipe/limit_breaker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/bin/bash
# Abort on Error
# code obtained from https://stackoverflow.com/questions/26082444/how-to-work-around-travis-cis-4mb-output-limit

set -e

export PING_SLEEP=30s
export WORKDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
export BUILD_OUTPUT=$WORKDIR/build.out

touch $BUILD_OUTPUT

dump_output() {
echo Tailing the last 500 lines of output:
tail -500 $BUILD_OUTPUT
}
error_handler() {
echo ERROR: An error was encountered with the build.
dump_output
exit 1
}
# If an error occurs, run our error handler to output a tail of the build
trap 'error_handler' ERR

# Set up a repeating loop to send some output to Travis.

bash -c "while true; do echo \$(date) - building ...; sleep $PING_SLEEP; done" &
PING_LOOP_PID=$!

# My build is using maven, but you could build anything with this, E.g.
# your_build_command_1 >> $BUILD_OUTPUT 2>&1
# your_build_command_2 >> $BUILD_OUTPUT 2>&1
conda build ./recipe >> $BUILD_OUTPUT 2>&1

# The build finished without returning an error so dump a tail of the output
dump_output

# nicely terminate the ping output loop
kill $PING_LOOP_PID
15 changes: 12 additions & 3 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,28 +13,34 @@ source:

build:
number: 0
skip: true # [win32 or osx or py34]
skip: true # [win32 or py34]
features:
- vc9 # [win and py27]
- vc10 # [win and py34]
- vc14 # [win and (py35 or py36)]

script: python setup.py install --single-version-externally-managed --record record.txt

requirements:
build:
- python
- gcc # [osx]
- setuptools
- blas 1.1 # [not win]
- openblas 0.2.20|0.2.20.* # [not win]
- numpy >=1.11.3
- scipy >=0.18.1
- six >=1.5.0
- smart_open >=1.2.1
run:
- python
- blas 1.1 # [not win]
- openblas 0.2.20|0.2.20.* # [not win]
- numpy >=1.11.3
- scipy >=0.18.1
- six >=1.5.0
- smart_open >=1.2.1

test:
requires:
- nose
Expand All @@ -58,10 +64,13 @@ test:
- gensim.topic_coherence
files:
- forge_test.sh
- limit_breaker.sh

commands:
- time bash ./forge_test.sh # [not win]
- nosetests --exe -v gensim # [win]

- conda inspect objects -p $PREFIX gensim # [osx]

about:
home: http://github.com/RaRe-Technologies/gensim
license: LGPL-3.0
Expand Down
4 changes: 4 additions & 0 deletions recipe/profiler.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import cProfile
from gensim.test.test_word2vec import *
import sys
cProfile.run("unittest.main()","combined-"+str(sys.version_info[0])+"."+str(sys.version_info[1]))
13 changes: 13 additions & 0 deletions recipe/test_fast.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import logging
import unittest
from gensim.models import doc2vec

class TestFastVersion(unittest.TestCase):
def testvesion(self):
"""Tests fast version"""
print("FAST VERSION value is ",doc2vec.FAST_VERSION)
self.assertTrue(doc2vec.FAST_VERSION!=-1)

if __name__ == '__main__':
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.DEBUG)
unittest.main()