Какие у тебя hard skills?

# Мои Hard Skills как Data Scientist

1. Программирование и языки

Python (Expert, 10+ лет):

• Production code: REST API, сервисы, микросервисы
• Data processing: NumPy, Pandas
• ML модель разработка
• Unit testing: pytest, unittest
• Type hints, clean code

SQL (Advanced, 8+ лет):

• PostgreSQL, MySQL, MongoDB, Elasticsearch
• Сложные запросы: JOINs, subqueries, window functions, CTEs
• Оптимизация (EXPLAIN, индексирование)
• Schema design, нормализация, relationships

Другие: R (базовый), Java (основы), Scala (базовый)

2. ML Frameworks и библиотеки

Scikit-learn (Expert):

• Классификация: Logistic Regression, SVM, Decision Trees, Naive Bayes
• Регрессия: Linear, Ridge, Lasso, Elastic Net
• Clustering: K-means, DBSCAN, Hierarchical Clustering
• Dimensionality reduction: PCA, t-SNE
• Preprocessing: StandardScaler, OneHotEncoder, PolynomialFeatures
• Pipeline construction для reproducibility
• Cross-validation: StratifiedKFold, TimeSeriesSplit, GroupKFold

XGBoost / LightGBM (Expert):

• Hyperparameter tuning: GridSearchCV, RandomizedSearchCV, Bayesian Optimization
• Feature importance analysis
• SHAP values для интерпретации моделей
• Handling imbalanced classes (scale_pos_weight, class_weight)
• Model monitoring и performance tracking

TensorFlow / Keras (Advanced):

• Dense layers, Conv2D, LSTM, GRU, Attention layers
• Transfer learning (ResNet, VGG, EfficientNet pre-trained)
• Model compilation, training loops, callbacks
• EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
• Custom loss functions и metrics
• Batch normalization, dropout

PyTorch (Advanced):

• Custom models через nn.Module
• Autograd и backward propagation
• RNN, LSTM, базовые Transformers
• DataLoaders, optimizers (Adam, SGD, AdamW)
• Device management (CPU/GPU)

3. Специализированные ML домены

NLP (Natural Language Processing):

• Text preprocessing: tokenization, stemming, lemmatization, stop words removal
• Embeddings: TF-IDF, Word2Vec, FastText, BERT, GPT
• Sentiment analysis (классификация текстов)
• Topic modeling (LDA)
• Named Entity Recognition (NER)
• Text classification end-to-end

Computer Vision:

• Image classification: CNN (ResNet, VGG, EfficientNet, MobileNet)
• Object detection: YOLO, Faster R-CNN, SSD
• Image preprocessing: normalization, augmentation (rotation, flipping, brightness)
• Transfer learning для CV задач
• Batch processing изображений

Time Series Forecasting:

• ARIMA, SARIMA (автоматическая туннинг параметров)
• Prophet (Facebook): быстрый baseline
• LSTM для долгосрочных прогнозов
• Seasonal decomposition
• Lag features, rolling statistics, trend detection

Recommender Systems:

• Collaborative filtering: User-User, Item-Item
• Matrix factorization: SVD, Non-negative Matrix Factorization
• Content-based filtering
• Hybrid approaches
• Evaluation метрики: NDCG, MAP, Precision@K

4. Обработка и инженерия данных

Data Manipulation:

• Pandas: groupby, merge, pivot, reshape operations
• NumPy: arrays, broadcasting, vectorized operations
• Data cleaning: handling missing values (imputation, deletion)
• Outlier detection (IQR method, Z-score, Isolation Forest)
• Duplicate detection и removal

Feature Engineering:

• Polynomial features, interaction terms
• Binning continuous variables
• Encoding categorical: LabelEncoder, OneHotEncoder, OrdinalEncoder
• Domain-specific features (business logic)
• Feature scaling: StandardScaler, MinMaxScaler, RobustScaler, Normalizer
• Feature selection (SelectKBest, RFE)

Big Data:

• Apache Spark (PySpark): DataFrame operations, SQL, aggregations
• Dask: parallel processing для больших датасетов
• Data pipeline concepts

5. Оценка и валидация моделей

Classification Metrics:

• Accuracy, Precision, Recall, F1-score
• ROC-AUC, Precision-Recall AUC
• Confusion matrix, classification report
• Threshold optimization для бизнес-требований

Regression Metrics:

• MAE, MSE, RMSE
• MAPE, RMSLE
• R² score, Adjusted R²

Validation Techniques:

• K-fold cross-validation
• Stratified k-fold для дисбалансированных классов
• Time series split для временных рядов
• Leave-One-Out CV

6. Развёртывание моделей (Deployment)

REST API Development:

• Flask: создание endpoints, routing
• FastAPI: async API, automatic documentation
• Model serialization: pickle, joblib, TensorFlow SavedModel, ONNX
• API documentation (Swagger/OpenAPI)

Containerization:

• Docker: Dockerfile, image building, optimization
• Docker Compose для multi-service приложений
• Container registry (Docker Hub, ECR)

Cloud Platforms:

• AWS: EC2, S3, CloudWatch, SageMaker (базовое)
• Google Cloud: AI Platform, Vertex AI (основы)
• Heroku для quick deployment

MLOps:

• MLflow: experiment tracking, artifact storage, model registry
• DVC: data versioning, pipeline management
• Model monitoring: performance degradation detection
• Automated retraining pipelines

7. Статистика и математика

Статистика:

• Hypothesis testing: t-test, chi-square, ANOVA, Mann-Whitney U
• Confidence intervals, p-values
• Statistical significance assessment
• A/B testing: power analysis, sample size calculation

Линейная алгебра:

• Vectors, matrices, operations
• Eigenvalues, eigenvectors (для PCA)
• Matrix decomposition: SVD, QR, Cholesky
• Matrix norms, conditioning

Вероятность:

• Distributions: Normal, Binomial, Poisson, Exponential
• Bayes theorem, conditional probability
• Expectation, variance, covariance

8. Визуализация

Libraries:

• Matplotlib: line plots, scatter, histograms, heatmaps
• Seaborn: statistical visualizations, violin plots, pair plots
• Plotly: interactive charts, 3D visualizations
• Bokeh: interactive plots

Skills:

• Creating interpretable visualizations for non-technical audience
• Dashboard creation (Grafana, Metabase basics)
• Data storytelling

9. Базы данных

Relational (SQL):

• Database design: normalization, relationships (1:N, M:N)
• Indexing strategy, query optimization
• ACID properties, transactions
• Views, triggers, stored procedures

NoSQL:

• Document databases: MongoDB
• Key-value stores: Redis (basics)

10. Контроль версий и сотрудничество

Git:

• Branching strategy: Git Flow, trunk-based development
• Merging, rebasing, cherry-pick
• Pull requests, code review process
• Conflict resolution

Collaboration:

• Project management: Jira, Trello, GitHub Issues
• Jupyter Notebooks для data exploration
• Stakeholder communication и presentation

Резюме: Уровни владения

EXPERT (10+ лет):

• Python, SQL, Scikit-learn, Data preprocessing
• Statistics, A/B testing, Feature engineering

ADVANCED (5-8 лет):

• XGBoost/LightGBM, TensorFlow/Keras
• Pandas, NumPy, Model evaluation
• Flask API, Docker

INTERMEDIATE (2-4 года):

• PyTorch, NLP, Computer Vision
• Time Series, AWS, MLflow/DVC

FOUNDATIONAL:

• Scala, Java, Apache Spark, GCP, Azure

---

Итого: 10+ лет профессионального опыта в машинном обучении и аналитике данных с фокусом на production системы.