deep neural networks

On training algorithms for neural networks

-

B9 L2 R2322 H1

In this talk, I will first give a convergence analysis of gradient descent (GD) method for training neural networks by relating them with finite element method. I will then present some acceleration techniques for GD method and also give some alternative training algorithms

Towards Designing Robust Deep Learning Models for 3D Understanding

-

B3 L5 R5220

Deep Neural Networks (DNNs) have shown huge success over the years to solve many 2D computer vision tasks driven by massive labeled 2D datasets and advancements in 2D vision models, but less success is witnessed on 3D vision tasks. This dissertation proposes innovative approaches to enhance the robustness of DNNs for 3D understanding and in 3D settings. The research focuses on two main areas: adversarial robustness on 3D data and setups, and the robustness of DNNs to realistic 3D scenarios. Two paradigms for 3D understanding are discussed: representing 3D as a set of 3D points and performing 2D processing of multiple images of the 3D data.

Deep Neural Network Approximation via Function Compositions

-

KAUST

Abstract Deep neural networks have brought transformative changes in areas such as image recognition and healthcare, yet their underlying theoretical principles remain somewhat elusive. The main emphasis of this presentation is to explore the theoretical dimensions of deep neural networks, with a special focus on evaluating their approximation capabilities. We begin by assessing the approximation errors in ReLU networks when they attempt to approximate various target functions, including (Lipschitz) continuous functions, polynomials, and smooth functions. Utilizing the concept of Vapnik

Approximation and Generalization Errors in Deep Neural Networks for Sobolev Spaces measured by Sobolev Norms

-

KAUST

Abstract In this presentation, we initially discuss the approximation capabilities of deep neural networks (DNNs) with ReLU and the square of ReLU as activation functions for Sobolev functions measured by the Sobolev norms \(W^{m,p}\) where \(m \ge 1\). Subsequently, we consider how to address the issue of the curse of dimensionality for DNNs’ approximation. Finally, we analyze the generalization errors associated with DNNs using such Sobolev loss functions. Additionally, we provide recommendations on when to opt for deeper NNs versus wider NNs, considering factors such as the number of sample