For a long time mathematics was divided in two main areas, those disciplines 
who could be applied in physics, and this was mainly the field of 
differential equations, and the more exoteric disciplines of pure mathematics.
Among pure mathematics the purest of all was considered number theory. In the 
last ten years, there have been discovered many connections between number 
theory and physics, mainly on the road of recent new insights in number 
theoretical aspects of topology. Some purely mathematical results were 
obtained by physicists.

A big book about the aspects of number theory and function theory relevant to 
these applications is 

Waldschmidt/Moussa/Luck/Itzkyson (ed.): From number theory to 
physics. Springer 1992, 680 p. ISBN 3-540-53342-7. DM 133.

Although originating from a meeting at the Centre de Physique in Les Houches 
in 1989, the book contains 14 long expository articles on the field, which 
usually start from scratch and are well written. 
A companion volume from the same conference is 

Luck/Moussa/Waldschmidt (ed.): Number theory and physics. 
Springer 1990, 300 p. ISBN 3-540-52129-1. DM 99. *4779.

It contains those seminars on the conference which concern the more 
concreteley physical aspects:

    Conformally invariant field theories and quantum groups;
      Quasicrystals;
      Spectral problems, automata, and substitutions;
      Dynamical and stochastic systems;
      Further arithmetical problems.

From the preface: "Number theory, or arithmetic, sometimes referred to as the 
queen of mathematics, is often considered as the purest branch of 
mathematics. It also has the false reputation of being without any 
application to other areas of knowledge. Nevertheless, throughout their 
history, physical and natural sciences have experienced numerous unexpected 
relationships to number theory. [...]
The most recent developments of theoretical physics have involved more and 
more questions related to number theory, and in an increasingly direct way. 
This new trend is especially visible in two broad families of physical 
problems. The first class, dynamical systems and quasiperiodicity, includes 
classical and quantum chaos, the stability of orbits in dynamical systems, 
KAM theory, and problems with small denominators, as well as the study of 
incommensurate structures, aperiodic tilings, and quasicrystals. The second 
class, which includes the string theory of fundamental interactions, 
completely integrable models, and  conformally invariant two-dimensional 
field theories, seems to involve modular forms and p-adic numbers in a 
remarkable way."