Developing surrogate geophysical models from data is a key research topic in atmospheric andoceanic modeling because of the large computational costs associated with numerical simulationmethods. Researchers have started applying a wide range of machine learning models, in particularneural networks, to geophysical data for forecasting without these constraints. However, constructingneural networks for forecasting such data is nontrivial and often requires trial and error. To that end,we focus on developing proper-orthogonal-decomposition-based long short-term memory networks(POD-LSTMs). We develop a scalable neural architecture search for generating stacked LSTMs toforecast temperature in the NOAA Optimum Interpolation Sea-Surface Temperature data set. Ourapproach identifies POD-LSTMs that are superior to manually designed variants and baseline time-series prediction methods. We also assess the scalability of different architecture search strategieson up to 512 Intel Knights Landing nodes of the Theta supercomputer at the Argonne Leadership Computing Facility.