Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci] [python] reduce unnecessary data loading in tests #3486

Merged
merged 16 commits into from
Oct 29, 2020
Merged

[ci] [python] reduce unnecessary data loading in tests #3486

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
79c7007
[ci] [python] reduce unnecessary data loading in tests
jameslamb Oct 26, 2020
f792015
add profiling files to gitignore
jameslamb Oct 26, 2020
c71a30f
just use cache()
jameslamb Oct 26, 2020
979b76a
default on cache size
jameslamb Oct 26, 2020
a86a6c2
patch lru_cache on Python 2.7
jameslamb Oct 26, 2020
afff7d3
linting
jameslamb Oct 26, 2020
c3067ee
Merge branch 'master' into misc/faster-tests
jameslamb Oct 27, 2020
25a9bc7
reduce duplicated code
jameslamb Oct 27, 2020
3af0a5c
missing warnings
jameslamb Oct 27, 2020
999c5b6
Merge branch 'master' into misc/faster-tests
jameslamb Oct 28, 2020
3394222
fix imports
jameslamb Oct 28, 2020
3a5fe60
fix lru_cache backport
jameslamb Oct 28, 2020
c1cb1b2
missing kwargs
jameslamb Oct 28, 2020
16ec8aa
Apply suggestions from code review
jameslamb Oct 28, 2020
4c2e7be
reduce duplicated code
jameslamb Oct 28, 2020
dfb0fd3
cache in test_plotting
jameslamb Oct 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,8 @@ htmlcov/
.coverage.*
.cache
nosetests.xml
prof/
*.prof
coverage.xml
*,cover
.hypothesis/
Expand Down
Empty file added tests/python_package_test/__init__.py
View file
Open in desktop
Empty file.
13 changes: 10 additions & 3 deletions tests/python_package_test/test_basic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,18 @@
from sklearn.datasets import load_breast_cancer, dump_svmlight_file, load_svmlight_file
from sklearn.model_selection import train_test_split

from .utils import lru_cache


@lru_cache(maxsize=None)
def _load_breast_cancer(**kwargs):
return load_breast_cancer(**kwargs)


class TestBasic(unittest.TestCase):

def test(self):
X_train, X_test, y_train, y_test = train_test_split(*load_breast_cancer(return_X_y=True),
X_train, X_test, y_train, y_test = train_test_split(*_load_breast_cancer(return_X_y=True),
test_size=0.1, random_state=2)
train_data = lgb.Dataset(X_train, label=y_train)
valid_data = train_data.create_valid(X_test, label=y_test)
Expand Down Expand Up @@ -86,7 +93,7 @@ def test(self):
os.remove(tname)

def test_chunked_dataset(self):
X_train, X_test, y_train, y_test = train_test_split(*load_breast_cancer(return_X_y=True), test_size=0.1, random_state=2)
X_train, X_test, y_train, y_test = train_test_split(*_load_breast_cancer(return_X_y=True), test_size=0.1, random_state=2)

chunk_size = X_train.shape[0] // 10 + 1
X_train = [X_train[i * chunk_size:(i + 1) * chunk_size, :] for i in range(X_train.shape[0] // chunk_size + 1)]
Expand Down Expand Up @@ -309,7 +316,7 @@ def check_asserts(data):
self.assertAlmostEqual(data.label[1], data.weight[1])
self.assertListEqual(data.feature_name, data.get_feature_name())

X, y = load_breast_cancer(return_X_y=True)
X, y = _load_breast_cancer(return_X_y=True)
sequence = np.ones(y.shape[0])
sequence[0] = np.nan
sequence[1] = np.inf
Expand Down
Loading