Recently there has been an increased interest in the development of kernel methods for learning with sequential data. The truncated signature kernel is a new learning tool designed to handle irregularly sampled, multidimensional data streams. In this article we consider the untruncated signature kernel and show that for paths of bounded variation it is the solution of a Goursat problem. This linear hyperbolic PDE only depends on the increments of the input sequences, doesn't require the explicit computation of signatures and can be solved using any PDE numerical solver; it is a kernel trick for the untruncated signature kernel. In addition, we extend the analysis to the space of geometric rough paths, and establish using classical results from stochastic analysis that the rough version of the untruncated signature kernel solves a rough integral equation analogous to the Goursat problem for the bounded variation case. Finally we empirically demonstrate the effectiveness of this kernel in two data science applications: multivariate time-series classification and dimensionality reduction.