<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="static/style.xsl"?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T21:43:45Z</responseDate><request verb="GetRecord" identifier="oai:uwspace.uwaterloo.ca:10012/23378" metadataPrefix="dim">https://dspacemainprd01.lib.uwaterloo.ca/server/oai/request</request><GetRecord><record><header><identifier>oai:uwspace.uwaterloo.ca:10012/23378</identifier><datestamp>2026-05-22T07:00:15Z</datestamp><setSpec>com_10012_9924</setSpec><setSpec>com_10012_1</setSpec><setSpec>col_10012_9926</setSpec><setSpec>col_10012_6</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:doc="http://www.lyncode.com/xoai" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author">George, Joshua Joseph</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2026-05-21T18:44:03Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2026-05-21T18:44:03Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2026-05-21</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="submitted">2026-05-19</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">https://hdl.handle.net/10012/23378</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract">We propose and analyze structure-preserving methods for first-order optimization of Lipschitz
smooth objectives by interpreting the dynamics as a dissipative Hamiltonian system, in which the
model parameters evolve jointly with an auxiliary momentum variable. This formulation induces
a natural energy dissipation mechanism that motivates the design of optimization algorithms that
inherit a discrete energy decay property. We develop discrete gradient (DG) methods that pre-
serve an exact discrete time energy decay property, ensuring monotone dissipation independent of
stepsize. Building on this framework, we introduce variants which empirically reduce oscillations,
improve runtime, and improve robustness to ill-conditioned problems.

To address the computational cost of the implicit DG methods, we propose semi-implicit
discrete gradient (SIDG) schemes obtained by linearizing the DG updates and incorporating
curvature through L-BFGS Hessian approximations, which are used to efficiently solve the result-
ing linear systems. These schemes retain key structure-preserving properties while significantly
reducing computational cost, yielding a practical balance between stability and efficiency. We es-
tablish monotone energy decay, boundedness of iterates, and sublinear convergence to first-order
stationary points.

Numerical experiments on ill-conditioned least-squares problems, regularized logistic regres-
sion, physics-informed neural networks, and CIFAR-10 image classification demonstrate good
performance despite ill-conditioning and competitive performance as compared to widely used
optimizers such as ADAM, Stochastic gradient descent, and L-BFGS.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso">en</dim:field>
   <dim:field mdschema="dc" element="publisher" lang="en">University of Waterloo</dim:field>
   <dim:field mdschema="dc" element="title">A Hamiltonian Systems Approach to Neural Network Optimization</dim:field>
   <dim:field mdschema="dc" element="type">Master Thesis</dim:field>
   <dim:field mdschema="dc" element="pending">false</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="department">Applied Mathematics</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="discipline">Applied Mathematics</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="grantor" lang="en">University of Waterloo</dim:field>
   <dim:field mdschema="uws-etd" element="degree">Master of Mathematics</dim:field>
   <dim:field mdschema="uws-etd" element="embargo" qualifier="terms">0</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="advisor">Morris, Kirsten</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="advisor">C. Del Rey Fern´andez, David</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="affiliation1">Faculty of Mathematics</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="city" lang="en">Waterloo</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="country" lang="en">Canada</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="province" lang="en">Ontario</dim:field>
   <dim:field mdschema="uws" element="typeOfResource" lang="en">Text</dim:field>
   <dim:field mdschema="uws" element="peerReviewStatus" lang="en">Unreviewed</dim:field>
   <dim:field mdschema="uws" element="scholarLevel" lang="en">Graduate</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>