Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
Machine Learning
ML RIT
| Question | Answer |
|---|---|
| What are the general descriptions/purposes of Data Science, Deep Learning, Machine Learning, and AI? | AI = planning, reasoning, and learning (biggest category), ML = learning only (next smallest), Deep Learning = ML with deep models (next smallest), Data Science = preprocess, pipeline, deploy (connected to deep learning) |
| Pipeline of training an ML model | 1. prep data, 2. choose features (characteristics of input), 3. train model, 4. evaluate model using metrics |
| ML definition | Model learns from data w/o being explicitly programmed --> learns from experience E on task T using performance measure P if performance in T measured by P improves with E --> so, if you add more data/info, you improve performance |
| Components of learning | input: x; output: y; target function: f = X --> y; Data: (x1, y1), ... , (xn, yn) |
| In ML, you know ____ and want to find ____. | You have an unknown target function and have input data (x and y or just x depending on learning style) and have to find f. |
| What are the two main types of ML problems? | 1. supervised learning (classification/regression) = have input and labels and find rule to predict output given new data; 2. unsupervised learning = given set of samples with no labels, group them into categories; |
| Classification vs. Regression | Classification = which category? Regression = how much? |
| Linear Regression formula | y = a + bx --> y = predicted val / output, a = intercept, b = slope / coefficient, x = input |
| Classification Problem | Given records (x1, y1), ... , (xn, yn) where xi = (xi1, ..., xip), and each record has attributes and a class label, find a model for the class label as a function of the attributes. |
| Binary vs. Multiclass Classification | Binary: y = {0, 1} --> choose btwn 2 categories --> Multiclass: y = {1, 2, ..., k} --> choose btwn k categories |
| Linear Classification | Find the best line to separate data into 2 categories --> during training, you find the best line equation and during testing, you see which side of the line the data falls |
| For a dataset, what are the minimum and maximum number of possible classes? How do you determine how many clusters to do? | min = 1 class total for all points; max = 1 class per point --> to determine # clusters, do it on your own since you don't have a y. Consider the domain and objectives to do so. |
| Feature transformation | modify existing features without adding more information --> same features, different representation |
| Feature engineering | create new features using domain knowledge --> adds new info |
| Are features and targets always the same data type? | no |
| Most learning algorithms only deal with ____ data. | numerical |
| Date types for features and targets | numerical (continuous/discrete #), integer, categorical (variables), binary (0, 1), text |
| What are the four ways to convert non-numeric data to numeric | 1. integer / label encoding 2. one-hot 3. ordinal 4. binary |
| How does integer encoding / label encoding work? | Assign a unique number to each category, where the unique numbers don't have any meaning beyond being unique identifiers. |
| How does one hot encoding work? | Create a matrix where in each column, everything is a 0 except for a 1 at the name of the feature. You need a large matrix to store all of the features. |
| How does binary encoding work | It is more compact than one hot since you use binary #'s to denote different categories (01, 10, 11, 01, etc.) --> needs logN columns for n categories --> 4 rows = 2 vars, 3 vars = 2^3 = 8, 5 vars = 2^5 = 32 |
| How does ordinal encoding work? | Map categories to integers while preserving order (i.e. poor = 1, good = 2, very good = 3, excellent = 4). |
| What is feature scaling? Why do you do feature scaling? | It transforms numerical features so they are on a similar scale. Since ML models compare numbers, it prevents large features from dominating small ones. Also, without scaling, can have slow or unstable training. |
| What types of scaling exist? | 1. min-max scaling, z-score normalization (standardization) |
| Min-Max Scaling formula and why it's beneficial | x_norm = (x - min(x))/(max(x) - min(x)) --> ensure features with large ranges don't dominate model, speed up algorithm convergence, standardize inputs so all features contribute more equally to output, helps avoid numerical instability --> 0 to 1 output |
| Z-Score Normalization (Standardization) | x_stand = (x - mean(x))/(Standard Deviation (x)) --> not exact 0 - 1 output boundary. Mostly falls within -3 to 3 |
| Goals of feature engineering | Convert context to input to learning algorithm, expose the structure of the concept to the learning algorithm, and work wll with the structure of the model the algorithm will create |
| Types of feature engineering | 1. tokenizing 2. bag of words 3. n-grams 4. tf-idf 5. embeddings |
| Tokenizing | break text into words and construct symbolic representations of the text |
| Bag of Words | represent each document as a fixed length numerical vector by counting the frequency of each token from the predefined vocabulary while ignoring word order and syntax |
| N-Grams | Construct sequences of N tokens (n-grams) to capture local context and word order info lost in unigram reps --> N = 1, each token 1 word long , N = 2, each is 2 words long, N = 3, each is 3 words long, etc. --> group tokens together to capture rel |
| TF-IDF | compute importance-weighted numerical features by combining local importance (TF) and global rarity (IDF) --> TF = num times term appears in doc / total num terms; IDF = log[(# docs)/(# docs containing t)] --> TF-IDF = TF * IDF |
| How do Word2Vec and FastText work? | Words have coordinates in N dimensions, and regions of space contain similar concepts --> can create features by avg vector across words or count in specific regions. Words are replaced with meanings to become dense representations --> text to embedding |
| Why do normalization? | Helps made model's job easier cuz no need to learn what is big/small for a feature. Some model types benefit more |
| In feature selection, what methods help you determine the optimal features to use? | 1. features: take top n most common features (stop words); 2. mutual info = take n that contain the most info about a target on a train set 3. Take N that improve accuracy most on a hold out set (greedy search or add / remove words) |
| What does mutual info measure? | How much knowing one variable reduces uncertainty about another --> if you have two variables and know x, you know y |
| When do we use the validation set and why use instead of test set? | It's used WHILE you train so you can find the best model. You use it instead of the test set since if you use test set to tune hyperparameters while training, it will overfit later on during testing |
| In linear regression, the prediction function can be written as: y_hat = _____ | B0 + B1x1 + B2x2 + ... + Bnxn --> here, y_hat = predicted value or score, while each x is an independent / predictor variable. The B values are linear coefficients. |
| What is the generalized form of the linear regression prediction function? You are expressing the linear score as ____? | y_hat = XB since if all of the B coefficients are stored as a matrix, you can make B0 = 1 and simplify the bias + sum of weighted input visible above. |
| When doing linear regression, which part of the prediction function do you already know and which part do you want to find? | You want to calculate B, the coefficient, since in supervised learning, you already know the input and the true output. |
| What are the formulas for the three regression error functions: MAE, MSE, RSS? | residual r = y - y_hat -->MAE = 1/n * sum(1 to n)[ |r|] --> MSE = 1/n (sum 1 to n) [r^2] --> RSS = (sum 1 to n)[r^2] |
| How do you find the best linear model for linear regression? | Find the line that minimizes the error |
| In Least Squares (OLS) Parameter Fitting, how do you find B? Here, B0 = intercept and remaining B values are feature weights. | 1. Add B as first column of x (all 1s) --> 2. Construct y_hat = B0 + B1X1 + ... + BnXn --> 3. RSS(B) = (y - XB)^T * (y - XB) --> 4. Calculate B_hat = (X^T * X)^(-1) * (X^T * y) |
| What are the assumptions and cons of Closed Form Ordinary Least Squares (OLS)? | Assume: lin rel btwn x and y --> Cons: outliers and multicollinearity (x and y too similar) can distort estimates, not robust or suitable for high-dimensional data |
Created by:
gunnikak