The main reasons why connections on fibre bundles play a prominent role in many contemporary field models lie in the fact that they enable us to deal with invariantly defined objects. Gauge theory shows clearly that this is a basic physical principle.
In gauge theory, connections on principal bundles are well known to provide the mathematical description of gauge potentials of the fundamental interactions. Furthemore, since the characteristic classes of principal bundles are expressed in terms of the gauge strength, one also meets the topological phenomena in classical and quantum gauge models, e. g., anomalies. All gauge gravitation models belong to the class of metric-affine theories where a pseudo-Riemannian metric and a linear (non-symmetric) connection on a world manifold are considered on the same footing as independent dynamic variables.
Though gauge theory has made great progress in describing fundamental interactions, it is a particular case of field theories on fibre bundles. Differential geometry of fibre bundles and formalism of jet manifolds give the adequate mathematical formulation of classical field theory, where fields are represented by sections of fibre bundles. This formulation also is applied to classical mechanics seen as particular field theory on fibre bundles over the time axis. In summary, connections are the main ingredient in describing dynamic systems on fibre bundles, and in Lagrangian and Hamiltonian machineries. Any dynamic equation on a fibre bundle, by definition, is a connection.
Jet manifolds provide the appropriate language for theory of (non-linear) differential operators and equations, the calculus of variations, and Lagrangian and Hamiltonian formalisms. For this reason, one usually follows the general definition of connections as sections of jet bundles in order to include them in a natural way in describing field dynamics.
In quantum field models, requiring theory of Hilbert and other infinite-dimensional linear spaces, the concept of a connection is rather new. It is phrased in algebraic terms as a connection on modules and sheaves. This notion is equivalent to the above mentioned geometric ones in the case of structure modules of fibre bundles. Extended to the case of modules over graded commutative and non-commutative algebras, it provides the language of supergeometry and non-commutative geometry.
L. Mangiarotti, G. Sardanashvily, Connections in Classical and Quantum Field Theory (WS, 2000)
G. Giachetta, L. Mangiarotti, G. Sardanashvily, Advanced Classical Field Theory (WS, 2009)