Chapter 8 SSL#

Chapter 9 Adversarial Attack#

14. Explain how to realize a targeted attack to a network?

    Add a term $e(y,y^{target})$ to the loss function of the non-targeted attack, meaning we want the attack result to be as close as possible to the target. $L(x)=-e(y,\hat{y})+e(y,y^{target})$²

    1. Precondition: Non-perceivable $x^*=argmin_{d(x^0,x)<\epsilon}{L(x)}$
    2. Non-Targeted Attack: Loss function: put a minus sign in front of the cross entropy between y and the correct answer ŷ. As the loss decreases, the cross entropy increases, and the attack succeeds. $L(x)=-e(y,\hat{y})$

15. For L2-norm and L-infinite, which one is stricter for the requirement of “Non-perceivable”?

    L-infinity. Judging the difference has to take human perception into account, i.e. domain knowledge. For example, the “distance” between images is best captured by their infinity norm, which considers where the change is largest. For the “non-perceivable” requirement, the L∞ norm is stricter: it only considers the largest element of the vector, so the other elements can stay closer to the original values, whereas the L2 norm averages over all elements, letting some of them drift slightly from the original values.

16. What is white-box attack and black-box attack?

    💻: White-box and black-box attacks are two different ways of attacking a machine learning model. Both can degrade the model’s performance, so defenses are needed to keep the model secure.

    • In a white-box attack, the attacker knows all the parameters and structure of the target model, including its inputs and outputs, and can access the model’s internal information directly. The attacker can then use that information to generate adversarial examples that mislead the model into wrong predictions.
    • In a black-box attack, the attacker can only access the model’s inputs and outputs, not its internal information. The attacker has to collect enough information by testing and querying the model in order to generate adversarial examples that fool it into wrong predictions.

17. Describe one way of black box attack?

    1. Transfer attack (proxy network): 💻: This attack exploits the similarity between different models: train a proxy model, then transfer the adversarial examples generated on it to the target model to fool it into wrong predictions. (One pixel attack, Universal Adversarial Attack)

    2. 💻: Gradient-based attack: the attacker queries the target model many times to obtain its outputs, uses this information to compute the target model’s gradients, and then generates adversarial examples. This requires computing gradients through backpropagation and is usually computationally expensive.

18. List at least four ways of passive defense.

    1. Filter(e.g. Smoothing)
    2. Image Compression
    3. Generator
    4. Randomization
    5. Adversarial Training: build the “attack process” into training, to train a model that is harder to break

Chapter 10 Explainable Artificial Intelligence (XAI)#

19. Why do we need Explainable ML?

    💻: Explainable machine learning matters in several ways:

    1. Improving the credibility, interpretability and acceptability of models: in many scenarios, people need to understand how a model works internally in order to assess its credibility, interpretability and acceptability.
    2. Improving model performance:
       • Models are usually trained on historical data, and if that data is biased, so is the model. Explainable ML helps us find biases in the data and algorithms, and improve the model.
       • Explainable ML helps us understand how the model works internally, so that we know why wrong answers occur and can improve the model.

20. What question the local explanation and global explanation want to answer is?

    • Local Explanation: Explain the Decision
    • Global Explanation: Explain the WHOLE MODEL

21. Elucidate what the local explanation is and give two or more methods of local explanation.

    💻: Local explanation explains a single sample or prediction, aiming to understand why the model made that prediction for the sample and which factors were decisive.

    👦:

    • Heat maps:
      • Gray patches: alter or delete each component; if the network’s output changes a lot, we know the component is important.
      • Saliency map: add noise to each pixel separately and plot the magnitude of the partial derivative of the loss with respect to each altered pixel x on a map to get the saliency map.
    • What each layer of the network does:
      • Dimensionality-reduction visualization: take vectors from the hidden layers and visualize them after dimensionality reduction.
      • Probing: use the output vectors of certain layers for some classification tasks.

22. How to do global explanation by using constraint from generator?

    • Connect an image generator to the image classifier.
    • The image generator takes a z as input and outputs an image; the image classifier takes this image as input and outputs a classification.
    • Find a z such that the X produced by the image generator, when fed into the image classifier, yields a y in which the score of a particular class is as high as possible.
    • Feed the $z^*$ found into the generator and look at the generated image.

23. Explain the idea of Local Interpretable Model-Agnostic Explanations (LIME)

    Using an interpretable model to mimic the behavior of an uninterpretable model.

Chapter 11 Transfer learning#

24. For NLP and CV, which layer can be transferred (copied)? What is the behind reason?

    • NLP: usually copy the last few layers. The lower layers of a pre-trained language model capture basic linguistic features such as morphology, grammar and semantics, while the higher layers capture more complex linguistic structure and semantics specific to the pre-training task. Different NLP tasks have similar linguistic structure (Human languages share some common characteristics.), while words, grammar and so on differ.
    • CV: usually copy the first few layers. In CV tasks, the lower layers of a pre-trained convolutional neural network (CNN) capture low-level visual features such as edges, corners and textures, while the higher layers capture more complex visual patterns. Low-level visual features are similar across CV tasks, while high-level visual patterns and tasks differ.

25. How to do multiple task learning while using a neural network with the multi-layer structure?

    1. Multi-Head Network: 💻: A common approach in which each task has its own output layer but the hidden layers are shared. Each output layer has its task’s loss function, and all parameters are trained with backpropagation. The benefit is that each task’s specific structure and features are preserved.
    2. Shared Layers: 💻: Multi-task learning by sharing parameters between hidden layers, which can be convolutional or fully connected layers. Sharing these layers across tasks lets them jointly learn general features, improving performance on multiple tasks.

26. What is the domain-adversarial training and how realize it?

    💻: Domain-adversarial training is a common transfer learning technique. Its goal is to counter the differences between domains with a domain classifier, improving the model’s generalization. Specifically, a domain classifier is added during training and the model is trained against it, so that it learns more robust and generalizable features.

    Domain-adversarial training can be implemented in the following steps:

    1. Define the domain classifier: a binary classifier that decides whether the input belongs to the source domain or the target domain.

    2. Define the classifier: a base classifier trained on source-domain data.

    3. Define the overall loss: it has two parts, the classifier loss and the domain classifier loss. The classifier loss trains on source-domain data, and the domain classifier loss decides which domain the input belongs to.

    4. Train the network: in each iteration, randomly sample a batch of source-domain and target-domain data, compute their classifier loss and domain classifier loss, and update the network parameters with backpropagation.

    5. 👦: Note that the domain classifier should end up failing, while the loss is still kept as small as possible.

27. How to realize the zero-shot learning by the attributes?

    💻: In attribute-based zero-shot learning, each object is associated with a set of attributes such as color, shape and texture. These attributes describe the object and can be used to recognize it, even if the model has never seen it before.

    To train zero-shot learning with attributes, first collect a set of labeled data containing labeled objects and their attributes. Then train a classifier that predicts an object’s attributes from image features.

    Once the model is trained, it can classify objects it has never seen by extracting the object’s features, predicting its attributes with the classifier, and assigning it to the right class. This extends the model’s reach, since it generalizes to unseen data without retraining.

28. How to realize the zero-shot learning by the attribute embedding?

    💻: Zero-shot learning with attribute embeddings is similar to using attributes, except that attributes are converted into embedding vectors instead of being used as raw values.

    In this approach, each attribute is first mapped into an embedding space. This is usually done by taking the attribute as input and computing its embedding with an embedding layer. These embeddings can be seen as feature representations of the attributes.

    Then, for each object, its attributes are converted into embedding vectors, for example by averaging the embeddings of the object’s attributes. This average vector is called the object’s attribute embedding.

    Finally, a conventional classifier such as an SVM or a neural network can take the object’s attribute embedding as input and predict the class. If the object’s attribute embedding is similar to a known class embedding, the object is assigned to that class.

    Compared with attribute-based zero-shot learning, attribute embeddings are more flexible, since different embedding spaces and methods can be used for better performance. The method can also handle multiple attributes directly, without considering the correlations between them.

29. Describe the method of “Attribute embedding + word embedding” for the zero-shot learning.

    💻: “Attribute embedding + word embedding” is a common zero-shot learning method. The basic idea is to classify by representing the attributes and names of objects or classes. Specifically, each class is first described by attributes, such as color, shape and material, which can be extracted automatically from existing samples or annotated by hand. Then the attribute vector and the word vector are combined into one complete vector representation. For an unseen class, its attributes and name are used to build its vector representation, and a classifier assigns it to the right class. The method needs enough attribute annotations and good attribute representations, but it applies to many tasks, such as image classification, object detection and natural language processing.

30. How to define the training loss of the zero-shot learning?

    $k-f(x^n)\times g(y^n)+max_{m\not=n}{f(x^n)\times g(y^m)}$

    k is a hyperparameter: the matching pair (first term) should be as close as possible, and mismatched pairs (second term) as far apart as possible.

Chapter 12 Network compressing#

31. List at least five types of way for network compressing.

    • Network Pruning
    • Knowledge Distillation
    • Parameter Quantization
    • Architecture Design
    • Dynamic Computation

32. Why train a large model and then do network pruning, not train a small network?

    It is widely known that smaller network is more difficult to learn successfully. (Small models are harder to train.)

33. How to understand this sentence “Pruning is an architecture search paradigm”?

    The Lottery Ticket Hypothesis: a large network can be seen as a combination of many small sub-networks. Training the large network amounts to training many small networks.

    Not every small network can be trained successfully; gradient descent may not find a good solution that lowers its loss. But among a large number of sub-networks, as long as one succeeds, the large network succeeds. So the more small networks a large network contains, the more lottery tickets you have bought: more tickets mean a higher chance of winning. That is why the larger a network is, the more likely it is to train successfully.

    Pruning, then, is finding the best one among the many sub-networks.

34. How to do a knowledge distillation? What is the temperature for softmax and the formulas?

    • 💻: Process:

      1. Train the large model (teacher model) on the training set and obtain the softmax values from its outputs (logits) and the true labels.
      2. Use the logits produced by the large model on the validation set and the softmax temperature T to get soft labels.
      3. Train the small model (student model) with the soft labels.

    • Before the softmax, divide the values by a hyperparameter $T$, which makes the softmax output smoother.

      $y_i^{'}= \frac{exp(y_i/T)}{\sum_j{exp(y_j/T)}}$

35. How to realize parameter quantization?

    1. Using less bits to represent a value
    2. Weight clustering
    3. Represent frequent clusters by less bits, represent rare clusters by more bits (Huffman coding: frequent values get shorter codes, rare values get longer codes)

36. How to realize the Depthwise Separable Convolution?

    1. Depthwise Convolution: considers the relationships within a channel

       Take a layer with N input neurons and M output neurons. To reduce the number of parameters, insert another layer with K neurons between N and M.

       The original single layer has M x N parameters; split into two layers, the first layer U has N x K parameters and the second layer V has K x M. If K is much smaller than M and N, U and V together have far fewer parameters than W.

    2. Pointwise Convolution: considers the relationships between channels

       A further step on top of the depthwise convolution, with the filter size restricted to $1\times1$.

37. What is the mathematical mechanism of the Depthwise Separable Convolution?

    Low rank approximation

38. List three ways of dynamic computation.

    1. Dynamically adjusting the network depth (Dynamic Depth)
    2. Dynamically adjusting the network width (Dynamic Width)
    3. Computation based on sample difficulty (letting the network decide its own width and depth)

Chapter 13 Life-long learning#

39. Explain what the full name of “CLIP” is, what the task used in CLIP is, and what the loss function used in CLIP.

    Note: CLIP is OpenAI work that Prof. Shouliang Qi shared in class.

    • Full name: Contrastive Language-Image Pre-training
    • Task: matching a piece of text with an image, building a link between vision and language so that the model can handle images and text naturally
    • Loss: a batch of images and their corresponding texts produce features $I_f$ and $T_f$; compute the matrix $I_f*T_f$, take the diagonal as the labels, and apply cross-entropy loss along rows and columns to get $L_i$ and $L_t$; their average is the loss

40. Try to compare the life-long learning and transfer learning while there are two tasks.

    • Transfer Learning: I can do task 2 because I have learned task 1.(We don’t care whether machine can still do task 1.)

      Transfer learning cares whether the skills learned on the first task help with the second ⇒ how well the new task is done

    • Life-long Learning: Even though I have learned task 2, I do not forget task 1.

      Life-long learning cares whether, after learning the second task, the machine can still solve the first one ⇒ how well the old task is done

41. Why Catastrophic Forgetting will happen?

    During gradient descent, the parameters move towards where the loss of the current task is small. But since the two tasks have different error surfaces, low loss corresponds to different parameters, which ultimately causes catastrophic forgetting.

42. What is the idea of selective synaptic plasticity? What is the loss function of selective synaptic plasticity?

    • Idea: Some parameters in the model are important to the previous tasks. Only change the unimportant parameters. When learning new parameters, the old, important parameters should change as little as possible; the new task should only change parameters that are unimportant to past tasks.

    • Loss:

      $L^{'}(\theta)=L(\theta)+\lambda\sum_i{b_i(\theta_i-\theta_i^b)^2}$

      $\theta^b$ denotes the parameters learned from previous tasks. An extra term is added to the original loss, wanting the newly learned $\theta_i$ to stay as close as possible to $\theta^b_i$, with a “guard” $b_i$ for each parameter saying “how strongly” we want them to stay close, i.e. how important the original parameter is.

43. List two kinds of way of Additional Neural Resource Allocation.

    • Progressive Neural Network: train some extra neurons for every new task, so the model parameters of previous tasks are never changed.
    • PackNet: allocate a large model from the start, and let each task use only some of the parameters; earlier tasks must not use up all the parameters, so later tasks can still use them.
    • CPG: Progressive Neural Network + PackNet ⇒ the CPG model can add new parameters, and for each new task only some of the parameters are kept trainable.

44. How to do memory replay by a generator?

    • For each task, train a generator that produces data for that task.
    • When training later tasks, train on the data of the current task together with pseudo data produced by the generators trained on previous tasks.

45. What is the curriculum learning?

    💻: In deep learning, curriculum learning is a method that helps the model learn faster by ordering the training samples. Unlike conventional random sampling, curriculum learning first trains on relatively easy samples, then gradually adds harder ones. This keeps the model from getting stuck on hard samples too early, improving training efficiency and generalization.

Chapter 14 Meta-learning (not a focus)#

46. What is Meta Learning?

    👦: Learning a learning method

    💻: Meta learning is a machine learning approach that aims to learn how to learn. Normally, a model is trained on a large amount of training data and then tested on test data. Meta learning aims to let the model learn from a few training samples, so that it generalizes better to new tasks or environments.

    👦: It can also be seen as a function F whose input is training data and whose output is a model.

47. List what can be learned through meta learning.

    Net Architecture, Initial Parameters, Learning Rate, Data augmentation, etc. (Almost anything can be learned.)

48. What are the three steps of meta-learning?

    1. Function with unknown (define a function with unknown parameters $\theta$)
    2. Define loss function for learning algorithm $F_\phi$
    3. Optimization³

49. How to define the loss function of meta-learning

    1. Use the learned learning algorithm $F_\phi$ to train on one task’s training data, obtaining the model $f_{\theta^{1*}}$.
    2. For each task, test the trained model $f_{\theta^{1*}}$ on that task’s (labeled) “test data”, compute the cross entropy between the predictions and the ground truth, and sum over the test data to get $l^1$. Keep testing the learning algorithm on “other tasks” of the “same kind” to get $l^2$, $l^3$…
    3. Sum the losses of the learning algorithm over all “learning tasks” to get the total loss $L(\phi)=\sum_{n=1}^N{l^n}$.

50. List two or three methods of optimization. (not a focus)

    Stochastic gradient descent, Adam, NAG, etc.

    The loss function $L(\phi)$ of the learning algorithm $F_\phi$ has been defined.

    • If $\frac{\partial L(\phi)}{\partial \phi}$ is differentiable, gradient descent can be used.
    • If not, use RL (Reinforcement Learning) or EA (Evolutionary Algorithm) and just brute-force train it.

51. Try to compare machine learning and meta-learning from the aspects of function, training, and loss.

    • Goal:
      • ML: find a function f that can do the task
      • Meta: find a learning algorithm F that can find f
    • Training Data:
      • ML: one task is enough; train on that task’s “training data”
      • Meta: train on several tasks, each “training task” having “training data + test data”
    • Loss:
      • ML: the sum of the losses over all the training⁴ data of one task
      • Meta: $l$ is the loss of one task, and $L$ is the sum of the losses over all tasks

52. What is the full name of MAML for learning initialization?

    Model-Agnostic Meta-Learning

53. Try to compare the different ways of initialization (MAML, transfer learning, domain adaption, self-supervised learning).

    💻: Each initialization approach has its own use cases and strengths in deep learning.

    MAML (Model-Agnostic Meta-Learning): in few-shot learning scenarios, MAML learns new tasks quickly within a meta-learning framework. Its initialization comes from training on multiple tasks, running gradient descent on each to optimize the model parameters and obtain a good set of initial parameters, which are then fine-tuned on the new task. MAML suits scenarios that need fast iterative optimization on new tasks.

    Transfer learning: transfer learning initializes a new target task with model parameters trained on a source task. Using knowledge learned on the source task speeds up training on the target task and improves generalization. Transfer learning suits scenarios where the source and target tasks share similar features.

    Domain adaptation: domain adaptation addresses the differences between two domains by mapping source-domain and target-domain data. The model can be initialized with source-domain data and then fine-tuned with target-domain data. Domain adaptation suits scenarios where the source and target data distributions differ.

    Self-supervised learning: self-supervised learning learns from natural properties of the data itself, avoiding the cost of labeling large amounts of data by hand. For example, it can learn good feature representations by predicting images or videos from different viewpoints or times. Its training data can be generated automatically, which suits scenarios where data is scarce.

    👦: MAML uses labeled data, so naturally it works better, while self-supervised learning uses unlabeled data. If the different tasks in meta learning are seen as different domains, meta learning can be seen as a form of domain adaptation.

54. Why MAML is good?

    ANIL (Almost No Inner Loop), Feature Reuse

55. There are some ways of optimizer, such as Adagrad, RMSprop, NAG, Adam, can we learn an optimizer or learning rate?

    Yes!

56. What is full name of NAS? Explain how to learning NAS through reinforcement learning?(not a focus)

    • Full name: Network Architecture Search (NAS)
    • How:
      • $\hat{\phi}=argmin_{\phi}L(\phi)$, $\phi$: the agent’s parameters
      • Build a reinforcement learning agent network (whose output is the network architecture parameters), build a network from the agent’s output, test that network’s accuracy, and use $-L(\phi)$ as the agent’s reward to update the agent.
      • Example:
        1. Use an RNN to output a network architecture (action)
        2. Build the network architecture
        3. Test the network’s accuracy (observation)
        4. Update the RNN

57. In Omniglot, there are 1623 characters and each has 20 examples. What is N and K in the “N-way K-shot classification?

    • N:1623, K: 20
    • N-ways K-shot classification: In each task, there are N classes, each has K examples.

58. What is prompt learning (optional)?

    Not covered in class. 💻: Prompt learning is a new self-supervised method in NLP that guides the model to learn a specific task by generating a task-related text prompt. In prompt learning, the user designs a set of relevant prompts by hand to help the model learn the task.

59. Can we learn the data augmentation?

    Yes! The method wasn’t explained in detail; roughly, build a graph and train to find the optimal path.

60. How do consider the relationship between general artificial intelligence (GAI) (Large model) and the domain or task specified artificial intelligence (Small model)?

    Open question