1 Introduction

Given a compact metric space (X, d), we denote the space of all continuous functions \(f:X\rightarrow \mathbb {R}\) by C(X). If \(k:X\times X\rightarrow \mathbb {R}\) is a continuous reproducing kernel, see e.g. [1] and [18, Chapter 4] for an introduction, then it is well-known that its reproducing kernel Hilbert space (RKHS) H satisfies \(H\subset C(X)\). Such RKHSs have a variety of applications in different fields, and in particular in statistics [4] and machine learning [5, 14, 18] they have become very popular. In these applications, it is often desirable that H is sufficiently large. One way to express this is the notion of universality [12, 17], which assumes that H is dense in C(X), as well as the closely related notion of characteristic kernels [13, 15], which can be used to define distances between probability measures [16]. While universality is sufficient to establish basic learning guarantees for RKHS-base learning algorithms, see e.g. [18, Chapters 6–8], it is often not fully satisfactory in the sense that one would ideally have even larger RKHS to work with, see e.g. [2, Section 2.3] and [9]. This raises the question, whether, for example, there exist RKHSs H, for which the inverse inclusion \( C(X)\subset H\) holds, see also [10] for a discussion of this question in view of the distances between probability measures mentioned above. Here we note that if X is infinite, such an RKHS H cannot have a continuous kernel since otherwise we would have \(H\subset C(X) \subset H\) and the norms of H and C(X) would be equivalent by a double application of the closed graph theorem. In other words, C(X) would be isomorphic to a Hilbert space, which is false, see e.g. [11, Example 1.11.25].

Obviously, this reasoning does not exclude RKHSs H with a discontinuous kernel and \(C(X) \subset H\). However, the following theorem shows that no such RKHSs exist in essentially all situations of practical interest.

Theorem 1.1

Let (X, d) be a compact metric space. If X is uncountable, then there exists no RKHS H on X such that \(C(X) \subset H\).

Finally, recall that so-called reproducing kernel Banach spaces (RKBS) have recently gained attraction, see e.g. [3, 8], partially because they somewhat naturally appear in the analysis of neural networks. In a nutshell, an RKBS E is a Banach space of functions \(X\rightarrow \mathbb {R}\) such that all point evaluation functionals are continuous on E. Obviously, every RKHS is an RKBS and C(X) is an example of an RKBS that is not an RKHS if X is infinite. Consequently, every RKHS can be embedded into an RKBS while Theorem 1.1 shows that the converse is false. In this sense, the class of RKBSs is strictly richer than the class of RKHSs.

2 Proof

Proof of Theorem 1.1

Let us assume that there was an RKHS H on X such that \(C(X) \subset H\). Denoting the kernel of H by k, we define

$$\begin{aligned} X_n := \bigl \{x\in X: k(x,x)\le n\bigr \} \end{aligned}$$

for all \(n\in \mathbb {N}\). Since \(X= \bigcup _{n\ge 1} X_n\), we then find an \(m\in \mathbb {N}\) such that \(X_{m}\) is uncountable. We write \(X_*:= {\overline{X_{m}}}\) for the closure of \(X_{m}\) in X. Moreover, we define

$$\begin{aligned} C_{\textrm{uc}}(X_{m}):= \bigl \{ f:X_{m}\rightarrow \mathbb {R}\,\,\bigl | \,\,f \text{ is } \text{ uniformly } \text{ continuous }\bigr \}. \end{aligned}$$

Then it is well known, see e.g. [7, p. 195], that for every \(f\in C_{\textrm{uc}}(X_{m})\), there is a unique, uniformly continuous \({\hat{f}}:X_*\rightarrow \mathbb {R}\) such that the restriction \({\hat{f}}_{|X_{m}}\) of \({\hat{f}}\) to \(X_{m}\) satisfies \({\hat{f}}_{|X_{m}} = f\). Since \(X_*\) is compact, \({\hat{f}}\) is bounded, and hence f is bounded. Consequently, we can equip \(C_{\textrm{uc}}(X_{m})\) with the supremum-norm \(\Vert \cdot \Vert _\infty \). In addition, each \({\hat{f}}\in C(X_*)\) is just the continuous extension of \(f\in C_{\textrm{uc}}(X_{m})\) from the dense subset \(X_{m}\subset X_*\) to \(X_*\) and using this, we can easily verify that this extension operation gives an isometric and linear operator

$$\begin{aligned} {{\hat{\cdot }}}\, : C_{\textrm{uc}}(X_{m})\rightarrow & {} C(X_*), \\ f\mapsto & {} {\hat{f}}. \end{aligned}$$

Moreover, the compactness of \(X_*\) ensures that every \(f\in C(X_*)\) is uniformly continuous, and hence this extension operator is surjective and thus actually an isometric isomorphism. In particular, \(C_{\textrm{uc}}(X_{m})\) is a Banach space. In addition, [11, Example 1.11.25] shows that \(C(X_*)\) is not reflexive since \(X_*\) is infinite, and consequently, \(C_{\textrm{uc}}(X_{m})\) is not reflexive, either.

Let us now recall from e.g. [1, p. 351] that the RKHS of the restricted kernel \(k_m:=k_{|X_{m}\times X_{m}}\) is given by

$$\begin{aligned} H_m:= \bigl \{ h_{|X_{m}}\,\, \bigl |\,\, h\in H\bigr \}. \end{aligned}$$

Since \(k_m\) is bounded by construction, namely \(k(x,x) \le m\) for all \(x\in X_{m}\), we know that the inclusion map \({{\,\textrm{id}\,}}:H_m\rightarrow \ell _\infty (X_{m})\) is well-defined and continuous with

$$\begin{aligned} \Vert h \Vert _\infty \le \sqrt{m}\,\Vert h \Vert _{H_m} \end{aligned}$$
(2.1)

for all \(h\in H_m\), see e.g. [18, Lemma 4.23], where \(\ell _\infty (X)\) denotes the space of all bounded functions \(X\rightarrow \mathbb {R}\).

Our next goal is to show \(C_{\textrm{uc}}(X_{m})\subset H_m\). To this end, we fix an \(f\in C_{\textrm{uc}}(X_{m})\). Since X is, as a metric space, a normal space, see e.g. [6, Theorem 2.6.1], Tietze’s extension theorem, see e.g. [6, Theorem 2.6.4], then gives a \(g\in C(X)\) with \(g_{|X_*} = {\hat{f}}\). By our initial assumption \(C(X)\subset H\), we then have \(g\in H\), and hence we find the desired

$$\begin{aligned} f = {\hat{f}}_{|X_{m}} = g_{|X_{m}} \in H_m. \end{aligned}$$

For later use, we note that the just established inclusion \(C_{\textrm{uc}}(X_{m})\subset H_m\) in combination with (2.1) yields

$$\begin{aligned} \Vert f \Vert _\infty \le \sqrt{m}\,\Vert f \Vert _{H_m} \end{aligned}$$
(2.2)

for all \(f\in C_{\textrm{uc}}(X_{m})\).

We now show that the inclusion map \({{\,\textrm{id}\,}}: C_{\textrm{uc}}(X_{m})\rightarrow H_m\) is continuous. To this end, we fix a sequence \((f_n)\subset C_{\textrm{uc}}(X_{m})\) that converges to some \(f\in C_{\textrm{uc}}(X_{m})\) with respect to \(\Vert \cdot \Vert _\infty \) and for which there exists an \(h\in H\) with \(f_n \rightarrow h\) with respect to \(\Vert \cdot \Vert _H\). Now, \(\Vert f_n-f \Vert _\infty \rightarrow 0\) implies \(f_n(x) \rightarrow f(x)\) for all \(x\in X\). In addition, convergence in an RKHS norm implies pointwise convergence by the very definition of RKHSs, and hence we have \(f_n(x) \rightarrow h(x)\) for all \(x\in X\). Consequently, we have \(f=h\), and the closed graph theorem therefore shows that the inclusion map \({{\,\textrm{id}\,}}: C_{\textrm{uc}}(X_{m})\rightarrow H_m\) is continuous. For the constant

$$\begin{aligned} K:= \Vert {{\,\textrm{id}\,}}: C_{\textrm{uc}}(X_{m})\rightarrow H_m \Vert <\infty , \end{aligned}$$

we thus have \(\Vert f \Vert _{H_m} \le K \Vert f \Vert _\infty \) for all \(f\in C_{\textrm{uc}}(X_{m})\).

By combining the latter estimate with (2.2), we now conclude that for all \(f\in C_{\textrm{uc}}(X_{m})\), we have

$$\begin{aligned} m^{-1/2}\, \Vert f \Vert _\infty \le \Vert f \Vert _{H_m} \le K \Vert f \Vert _\infty . \end{aligned}$$

In other words, \( \Vert \cdot \Vert _{H_m}\) is an equivalent norm on \(C_{\textrm{uc}}(X_{m})\), and since \(\Vert \cdot \Vert _\infty \) is complete on \(C_{\textrm{uc}}(X_{m})\), so is \( \Vert \cdot \Vert _{H_m}\). Consequently, \(C_{\textrm{uc}}(X_{m})\) is isomorphic to a Hilbert space and thus reflexive. This contradicts our previous finding that \(C_{\textrm{uc}}(X_{m})\) is not reflexive. \(\square \)

Finally, we like to remark that the following theorem can be shown analogously.

Theorem 2.1

Let X be an uncountable set. Then there exists no RKHS H on X such that \(\ell _\infty (X) \subset H\).