Text this: A deep neural network for general scattering matrix