[PMLE CERTIFICATE - EXAMTOPIC] DUMPS Q13-Q16

EXAMTOPIC DUMPS Q13-Q16; Strategies to prevent Overfitting, Deteriorated model Accuracy, Input pipeline performance for Image data, Difference between CNNs and RNNs

Q 13.

You have trained a deep neural network model on Google Cloud. The model has low loss on the training data, but is performing worse on the validation data. You want the model to be resilient to overfitting. Which strategy should you use when retraining the model?

Strategies to prevent Overfitting

• ❌ A. Apply a dropout parameter of 0.2, and decrease the learning rate by a factor of 10.
  → Decrease the lr might training process slower and force the model to go though more epochs than necessary thus memorizing (overfitting) and not generalizing
• ❌ B. Apply a L2 regularization parameter of 0.4, and decrease the learning rate by a factor of 10.
• ⭕ C. Run a hyperparameter tuning job on AI Platform to optimize for the L2 regularization and dropout parameters.
• ❌ D. Run a hyperparameter tuning job on AI Platform to optimize for the learning rate, and increase the number of neurons by a factor of 2.
  → Increasing the num of neurons model larger and converge faster but easily start overfitting
  → TO RESOLVE THE OVERFITTING PROBLEM, BETTER TO REDUCE THE CAPACITY OF THE NETWORK : SMALLER NETWORK

L1, L2, Dropout, Epochs, Learning rate

Regularization
정규화 없이도 모델이 예측할 수 있는 지 확인한 후, 과적합 문제가 확인되는 경우에만 Regularization 정규화한다. 선형 모델과 비선형 모델은 정규화 방법이 다르다.
(1) linear model 선형모델 정규화
→ 모델의 size 크기를 줄이기 위해 : L1 regularization
→ 모델의 stability 안정성을 높이기 위해 : L2 regularization (_model's stability makes your model training more reproducible)_
→ regularization rate $λ$ starting from 1e-5 적절한 값을 찾는다.
_(2) Deep neural network model 정규화 : Dropout regularization 고정된 비율의 뉴런 랜덤으로 삭제 removes a random selection of a fixed percentage of the neurons in a network layer for a single gradient step. Dropout has a similar effect to L2 regularization. _
_ → Dropout rate : between 10% and 50% _

Strategies to prevent overfitting
Get more training data. (The best) 더 많은 데이터에서 학습한 모델이 일반화 성능이 더 좋다.
Reduce the capacity of the network. ⇒ Start with a small model = with a small number of learnable parameters determined by the number of layers and the number of units per layer (The simplest) 모델 규모를 축소 = 학습 가능한 파라미터 수를 줄인다. model's capacity⇒memorization capacity 모델의 파라미터는 nn의 경우 LAYER의 수와 LAYER UNIT의 개수에 의해 결정되고,
→ (1) 모델의 복잡도 (Tiny < Small < Medium < Large)에 따른 Loss 변화 차이 Large 첫 번째 에포크 이후 과대적합 시작, 큰 모델일수록 더 빠르게 Loss를 감소시키며 훈련 세트를 학습시킬 수 있지만 쉽게 과적함됨을 확인할 수 있다. (Large Train vs Large Val)
Add weight regularization : Placing constraints on the quantity and type of information your model can store. 정규화는 모델이 저장하는 정보의 양과 유형에 제약을 부과함으로써, 모델이 적은 수의 패턴을 기억해 최적화 과정동안 일반화 가능성이 높은 중요한 패턴에 초점 두고 학습하도록 만든다.
Add dropout.
Appropriate number of epochs 모델을 너무 오래 훈련하면 과적합될 수 있다.
Batch normalization
Learning Rate : Since the learning rate is the speed with which we move down the loss gradient, decreasing it will make this rate slower and force the model to go though more epochs than necessary thus memorizing (overfitting) and not generalizing.

Q 14.

You built and manage a production system that is responsible for predicting sales numbers. Model accuracy is crucial, because the production model is required to keep up with market changes. Since being deployed to production, the model hasn't changed; however the accuracy of the model has steadily deteriorated. What issue is most likely causing the steady decline in model accuracy?

Deteriorated model Accuracy

• ❌ A. Poor data quality
• ⭕ B. Lack of model retraining
  → Retraining is needed as the market is changing.
• C. Too few layers in the model for capturing information
• ❌ D. Incorrect data split ratio during model training, evaluation, validation, and test

Q 15.

You have been asked to develop an input pipeline for an ML training model that processes images from disparate sources at a low latency. You discover that your input data does not fit in memory. How should you create a dataset following Google-recommended best practices?

Input Image data pipeline - low latency & not fit in memory problem

• A. Create a tf.data.Dataset.prefetch transformation.
• ❌ B. Convert the images to tf.Tensor objects, and then run Dataset.from_tensor_slices().
  → tf.data.Dataset` is for IN-MEMORY
• ❌ C. Convert the images to tf.Tensor objects, and then run tf.data.Dataset.from_tensors().
  → tf.data.Dataset` is for IN-MEMORY
• ⭕ D. Convert the images into TFRecords, store the images in Cloud Storage, and then use the tf.data API to read the images for training.

• tf.data.Dataset is for data in memory.
• tf.data.TFRecordDataset is for data in non-memory storage
  • Store image, video, audio and unstructured data on Cloud Storage ; Store these data in large container formats on Cloud Storage. This applies to sharded TFRecord files if you're using TensorFlow, or Avro files if you're using any other framework.

Summary of the best practices for designing performant TensorFlow input pipelines

Full docs

* Use the prefetch transformation to overlap the work of a producer and consumer

* Parallelize the data reading transformation using the interleave transformation

* Parallelize the map transformation by setting the num_parallel_calls argument

* Use the cache transformation to cache data in memory during the first epoch

* Vectorize user-defined functions passed in to the map transformation

* Reduce memory usage when applying the interleave, prefetch, and shuffle transformations

Q 16.

You are an ML engineer at a large grocery retailer with stores in multiple regions. You have been asked to create an inventory prediction model. Your model's features include region, location, historical demand, and seasonal popularity. You want the algorithm to learn from new inventory data on a daily basis. Which algorithms should you use to build the model?

Difference between CNNs and RNNs

• ❌ A. Classification
• ❌ B. Reinforcement Learning
  → Reinforcement Learning is basically agent - task problems.
• ⭕ C. Recurrent Neural Networks (RNN)
  → RNN is the "Preferred algorithm for sequential data like time series, speech, text, financial data, audio, video, weather and much more" since "It learns over time what information is important and what is not" because they "can remember important things about the input they received, which allows them to be very precise in predicting what’s coming next".
• ❌ D. Convolutional Neural Networks (CNN)