Erdős–Hajnal Problem for H-Free Hypergraphs

Abstract

This paper deals with the minimum number $m_H(r)$ of edges in an H-free hypergraph with the chromatic number more than r. We show how bounds on Ramsey and Turán numbers imply bounds on $m_H(r)$.

1 Introduction

A hypergraph $G = (V, E)$ consists of a finite set of vertices V and a family E of subsets of V, which are called edges. A hypergraph G is called n-graph if every edge in G has size n. A vertex r-coloring of a hypergraph $G = (V, E)$ is a map from V to $\{1, \dots , r\}$. A coloring is proper if there are no monochromatic edges, i.e. any edge $g \in E$ contains two vertices of different colors. The chromatic number of a hypergraph G is the smallest number $\chi (G)$ such that there exists a proper $\chi (G)$-coloring of G. A vertex subset is independent in G if it does not contain an edge of G. The independence number of G is the maximal size $\alpha (G)$ of an independent set.

Although the subjects are defined and studied for n-graphs, this paper deals with 3-graphs only. For convenience, we restrict cited results on 3-graphs despite most of them being more general.

Let us start with best known bounds for 3-graphs without restrictions. Let m(r) be the smallest number of edges in a 3-graph with the chromatic number at least $r + 1$. Then

$$\begin{aligned} (1+o(1)) \frac{4}{e^2} r^3 \le m(r) \le \left( {\begin{array}{c}2r+1\\ 3\end{array}}\right) = (1+o(1)) \frac{4}{3}r^3, \end{aligned}$$

see [2] for details. See also the survey on related problems on n-graphs [10].

A subgraph of a 3-graph $G = (V,E)$ is a 3-graph $G' = (V',E')$, where $V' \subset V$, $E' \subset E$ and $E' \subset \left( {\begin{array}{c}V'\\ 3\end{array}}\right)$. An induced subgraph of a 3-graph $G = (V,E)$ is a 3-graph $G[V']$, formed from a vertex set $V' \subset V$ and all of the edges from G contained in $V'$. An H-free 3-graph is a 3-graph that does not contain H as a subgraph.

Bohman, Frieze and Mubayi [1] introduced the quantity $m_H(r)$, which is the minimal number of edges in an H-free 3-graph with chromatic number more than r. First of all, note that sometimes $m_H(r)$ is undefined. More precisely, this is the case iff H is a hyperforest, i.e. every two edges of H share at most one vertex and H has no cycles. We discuss it with more details later.

Structure of the Paper. The rest of the introduction is devoted to the known results, our contribution and classical tools. Section 1.1 contains known bounds on $m_H(r)$, mostly from paper [1]. Our results are listed in Sect. 1.2, where we show some relations $m_H(r)$ has with Ramsey and Turán numbers. The tools which we use in proofs of these relations are recalled in Sect. 1.3. The correctness of the definition of $m_H(r)$ is discussed in Sect. 1.4. In Sect. 2 we give proofs of the results. In Sect. 3 we list some open questions.

1.1 Known Bounds

Definition 1

Let H be a 3-graph. Define the balance $\rho (H)$ of H as follows

$$\begin{aligned} \rho (H):= \max _{H'} \frac{e' - 1}{v' - 3}, \end{aligned}$$

where $H'$ is a (nonempty) subgraph of H with $v'$ vertices and $e'$ edges. We say that H is balanced if this maximum occurs for H.

Theorem 1

([1]) If H is a balanced 3-graph with $\rho (H) > \frac{1}{2}$, then

$$\begin{aligned} m_H(r) = O \left( (r^2 \log r)^{\frac{3\rho -1}{2\rho -1}} \right) . \end{aligned}$$

In the following case an H-free 3-graph is also called a simple or linear 3-graph.

Theorem 2

([1, 7]) Let H be the 3-graph on 4 vertices with the set of edges $\{ \{1,2,3\},\{1,2,4\} \}$, then

$$\begin{aligned} m_H(r) = \theta \left( r^4 \log ^2 r \right) . \end{aligned}$$

In the following case an H-free 3-graph is also known as a graph with independent neighbourhoods. The upper bound is a corollary of Theorem 1.

Theorem 3

([1]) Let H be the 3-graph on 5 vertices with the set of edges $\{ \{1,2,3\},\{1,2,4\}, \{1,2,5\}, \{3,4,5\} \}$, then

$$\begin{aligned} c r^{\frac{7}{2}} \le m_H(r) \le C r^{\frac{7}{2}} \log ^\frac{7}{4}r. \end{aligned}$$

Hypergraph H in the following theorem is also known as $K^3_4$; further we denote it by $K_4$ since we are dealing only with 3-graphs.

Theorem 4

([1]) Let H be the 3-graph on 4 vertices with the set of edges $\{ \{1,2,3\},\{1,2,4\}, \{1,3,4\}, \{2,3,4\} \}$, then

$$\begin{aligned} m_H(r) = O \left( r^3 \log ^3 r \right) . \end{aligned}$$

1.2 Results

1.2.1 Relation with Ramsey Numbers

A Ramsey number $R(H, K_t)$ is the minimal number of vertices in a 3-graph G that guarantees existence of a copy of H or an independent set on t elements in G. We show that bounds on Ramsey numbers imply bounds on $m_H(r)$.

Theorem 5

Consider a 3-graph H. Let $a, b > 1$ and $A, B > 0$ be numbers such that for all $t \ge 1$

$$\begin{aligned} At^{a} < R(H,K_t) \le Bt^{b}. \end{aligned}$$

(1)

Then for all $r \ge 1$

$$\begin{aligned} B'r^{\frac{b}{b-1} + 2} \le m_H(r) \le A'r^{\frac{3a}{a-1}}, \end{aligned}$$

where

$$\begin{aligned} A' = \frac{(1+o(1))^{3a}}{6A^{\frac{3}{a-1}}}, \qquad B' = \left( \frac{(1-2^{1-b})}{2^{\frac{b}{b-1}+b+3}(12e)^{b-1} B \bigl (\frac{b}{b-1}+2\bigr )^b} \right) ^{1 / (b - 1)}. \end{aligned}$$

In [8] it was shown that for all $t \ge 1$

$$\begin{aligned} \frac{at^{3/2}}{(\log t)^{3/4}} \le R(C_3,K_t) \le bt^{3/2}, \end{aligned}$$

where $C_3$ is a loose cycle with 3 edges, i.e. the 3-graph on 6 vertices with the set of edges $\{\{1, 2, 3\}, \{3, 4, 5\}, \{5, 6, 1\}\}$, and $a, b > 0$ are some constants. A straightforward application of Theorem 5 gives

Corollary 1

There is a constant $C > 0$ such that for all $r \ge 1$

$$\begin{aligned} Cr^{5} \le m_{C_3}(r). \end{aligned}$$

However, the upper bound can be made more precise than the one implied by Theorem 5.

Proposition 1

There is a constant $C > 0$ such that for all $r \ge 1$

$$\begin{aligned} m_{C_3}(r) \le Cr^{5}(\ln r)^{5/2}. \end{aligned}$$

Theorem 6

Suppose that for a 3-graph H and some constants $A, B > 0$ and $b > 1$ we have

$$\begin{aligned} t^{At} < R(H,K_t) \le t^{Bt^b}. \end{aligned}$$

(2)

Then

$$\begin{aligned} r^3 \ln ^{1/b-o(1)} r \le m_H(r) \le (1+o(1)) \frac{8}{A^3} r^3 \left( \frac{\ln r}{\ln \ln r}\right) ^{3}. \end{aligned}$$

Let $K_4^{-}$ be the 3-graph with four vertices and three edges. In [5] it was stated that

$$\begin{aligned} R(K_4^-, K_t) = t^{\Theta (t)}. \end{aligned}$$

Then Theorem 6 immediately yields

Corollary 2

For some constant $C > 0$,

$$\begin{aligned} r^3 \ln ^{1-o(1)} r \le m_{K_4^-}(r) \le C r^3 \left( \frac{\ln r}{\ln \ln r}\right) ^3. \end{aligned}$$

The following bounds are from [4]:

$$\begin{aligned} 2^{c_1 n\log n} \le R (K_4,K_n) \le 2^{c_2n^2\log n} \end{aligned}$$

for some constants $c_1, c_2 > 0$ and any sufficiently large n. A direct application of Theorem 6 gives

Corollary 3

For some constant $C > 0$,

$$\begin{aligned} r^3 \ln ^{1/2-o(1)} r \le m_{K_4}(r) \le C r^3 \left( \frac{\ln r}{\ln \ln r}\right) ^3. \end{aligned}$$

Moreover, using the same idea of graph reduction by coloring some big independent set in the same color, as in proof of theorems of this section (but with much less effort), we obtain

Proposition 2

Let H be the hypergraph on 7 vertices with the set of edges $\{\{1,2,3\},\{1,4,5\},\{1,6,7\},\{2,4,6\} \}$. Then

$$\begin{aligned} m_H(r) > Cr^4, \end{aligned}$$

where $C = e/288$.

1.2.2 Relation with Turán Numbers

A Turán number $\textrm{ex}(n,H)$ is the maximum number of edges in an H-free 3-graph with n vertices. As with Ramsey numbers, we can obtain bounds for $m_H(r)$ from bounds on Turán numbers.

Theorem 7

Let H be a 3-graph and let $C_1 > 0$ be a constant such that for all $n \ge 1$

$$\begin{aligned} \textrm{ex}(n,H) \le C_1n^b. \end{aligned}$$

(3)

Then for any constant $C_2 > 0$ there exists some $R = R(C_2)$ such that for any $r > R$

$$\begin{aligned} m_H(r) > \frac{C_2r^\frac{2b}{b-1}}{(\ln \ln r)^{1+\frac{2b}{b-1}}}. \end{aligned}$$

The following proposition is folklore.

Proposition 3

Let H be a 3-graph, such that $t=|V(H)|=|E(H)|+2$. Suppose that there is an ordering of edges $h_1,\dots ,h_{t-2}$ such that $h_i=\{a_i,b_i,c_i\}$ for each $i>1$, where $\{a_i,b_i\}\subset h_{j(i)}$ for some ${j(i)}<i$ and $c_i\not \in h_j$ for $j<i$. Then

$$\begin{aligned} ex(n,H)\leqslant (t-3)\left( {\begin{array}{c}n\\ 2\end{array}}\right) . \end{aligned}$$

Note that such a graph H is balanced with balance 1, so Theorems 7 and 1 give

Corollary 4

Suppose that 3-graph H satisfies the conditions of Proposition 3. Then for some positive constants $C_1, C_2 > 0$ and for any sufficiently large r

$$\begin{aligned} C_1 \frac{r^4}{(\ln \ln r)^5}< m_H(r) < C_2 r^4 (\ln r)^2. \end{aligned}$$

1.3 Tools

1.3.1 Applications of LLL

We are going to use the following classical technique.

Lemma 1

(Symmetric Lovász local lemma) Let $A_1,\ldots ,A_n$ be events with probabilities $\textbf{P}(A_i) \le p$ for $1 \le i \le n$. Suppose that each event $A_i$ is mutually independent of all the other events except at most d of them. If $ep(d+1) < 1$, then $\textbf{P}(\wedge _{i=1}^n \overline{A}_i) > 0$.

The following lemma is a folklore corollary of Lemma 1.

Lemma 2

Let G be a 3-graph. Let $r>0$ be some integer. Suppose that for every $v\in V(G)$

$$\begin{aligned} \deg v \le \Delta = \frac{r^2}{3e}. \end{aligned}$$

Then G can be properly colored with r colors.

Proof

Let us assign to each vertex one of r colors at random uniformly and independently. For each $f \in E$ consider an event $A_f = ``f\in E\text { is monocromatic''}$. The probability of such event is $1/r^2 =: p$. Clearly each event $A_f$ is independent of $\{A_{g} :f \cap g = \emptyset \}$. There are at most $3(\Delta -1) + 1 < 3\Delta$ events (including $A_f$ itself) that are not in this family, so

$$\begin{aligned} ep(d+1) < \frac{3e\Delta }{r^2} = 1, \end{aligned}$$

and, by Lemma 1, G has some coloring in r colors with no monochromatic edges.

Lemma 2 allows us to care only about vertices with big degree when we are coloring some graph in r colors. Let us make this statement more precise.

Definition 2

Denote $c = (12e)^{-1}$. Consider an integer r and a 3-graph G. Define

$$\begin{aligned} V_{small}= & {} \{v \in V(G) \mid \deg v \le cr^2\}, \\ V_{big}= & {} V(G) \setminus V_{small}. \end{aligned}$$

By a direct application of Lemma 2 we obtain

Lemma 3

Let r be an integer and G be a 3-graph. Then $G[V_{small}]$ is $\lceil r/2\rceil$-colorable.

1.3.2 Greedy Coloring

The following approach was introduced by Pluhár [9]. Let us consider an arbitrary linear order $\pi$ on the vertex set of a given 3-graph G. Now one can go through vertices in order $\pi$ and color each considered vertex into the minimal color which does not create a monochromatic edge at this moment. By the construction, the result is a proper coloring f in some number of colors.

Let an r-chain be a set of edges $g_1,\dots , g_r$ such that $|g_i \cap g_j| = 1$ if $|i-j| = 1$, and 0 otherwise. We say that an r-chain is ordered in $\pi$ if $i < j$ implies that every vertex of $g_i$ is no bigger that every vertex of $g_j$ with respect to $\pi$.

Pluhár noted that the number of colors in f is the maximal size of an ordered chain plus one. In particular, if f uses more than r colors, then G contains an r-chain (in fact, a lot of r-chains). Then Gyárfás and Lehel [6] extended the latter statement as follows. If a greedy coloring of a 3-graph G uses more than r colors, then G contains a copy of every hypertree (connected hyperforest) with r edges.

1.4 Correctness

We have claimed earlier that the function $m_H(r)$ is properly defined for all H except hyperforests. Here we provide a brief proof of this statement. Indeed, if H is a hyperforest, then, as was discussed in the previous subsection, any H-free hypergraph can be properly colored in |E(H)| colors using greedy coloring method, so $m_H(r)$ is not defined for $r > |E(H)|$.

Otherwise, H contains either a pair of edges intersecting by two vertices, or a loose cycle $C_l$ for some $l \ge 3$, i.e. a 3-graph on 2l vertices with the set of edges $\{\{1, 2, 3\}, \{3, 4, 5\}, \dots , \{2\,l - 1, 2\,l, 1\}\}$. In the first case any simple 3-graph is H-free, so, in view of Theorem 2, $m_H(r) = O(r^4 \log ^2 r)$. For a graph with a loose cycle $C_l$ one may count the balance of the cycle as follows

$$\begin{aligned} \rho (C_l) = \frac{l - 1}{2l - 3} = \frac{1}{2} + \frac{1}{4l - 6}. \end{aligned}$$

The maximum in the definition of the balance is achieved on the whole cycle, so $C_l$ is balanced with the balance greater than $\frac{1}{2}$. Thus one can apply Theorem 1 to obtain the inequality

$$\begin{aligned} m_H (r) \le m_{C_l} (r) = O \left( (r^2 \log r)^{\frac{3\rho -1}{2\rho -1}} \right) = O(r^{2l}\log ^l r). \end{aligned}$$

2 Proofs

2.1 Proof of Theorem 5

2.1.1 The Upper Bound

The lower bound on $R(H, K_t)$ states that for each $t\ge 1$ there is an H-free 3-graph G with $|V(G)| = At^{a}$ and the independence number $\alpha (G) < t$. The chromatic number of G satisfies

$$\begin{aligned} \chi (G) \ge \frac{|V(G)|}{\alpha (G)} \ge At^{a-1}. \end{aligned}$$

Now for a given r we choose integer t such that $A(t-1)^{a-1} \le r < At^{a-1}$. Then

$$\begin{aligned} |E(G)| \le \frac{|V(G)|^{3}}{6}{} & {} = \frac{A^3t^{3a}}{6} = \frac{A^{3-\frac{3a}{a-1}}}{6} (At^{a-1})^{\frac{3a}{a-1}} \\{} & {} < \frac{A^{3-\frac{3a}{a-1}}}{6} \biggl (A\Bigl (\bigl (1+o(1)\bigr )(t-1)\Bigr )^{a-1}\biggr )^{\frac{3a}{a-1}}\le \frac{(1+o(1))^{3a}}{6A^{\frac{3}{a-1}}} \cdot r^{\frac{3a}{a-1}}. \end{aligned}$$

2.1.2 The Lower Bound

Suppose that G is an H-free 3-graph with $|E(G)| < B'r^{\frac{b}{b-1}+2}$ that is not r-colorable. Let such r be minimal possible.

From Lemma 3 it follows that $G[V_{big}]$ cannot be colored with no more than r/2 colors. Then

$$\begin{aligned} |E(G[V_{big}])| \ge \frac{B'}{2^{\frac{b}{b-1}+2}} \cdot r^{\frac{b}{b-1}+2}, \end{aligned}$$

(4)

Let $V_{big} = V_0 \cup V_1 \cup V_2 \cup \dots$, where

$$\begin{aligned} V_k = \{ v \in V(G) \mid 2^k cr^2 \le \deg v < 2^{k+1}cr^2\}, \end{aligned}$$

and $c = (12e)^{-1}$ is from the definition of $V_{big}$.

Suppose there is some k for which

$$\begin{aligned} \alpha (G[V_k]) \ge \frac{B'}{2^{k-1}cr^2}\bigl (r^{\frac{b}{b-1} + 2} - (r-1)^{\frac{b}{b-1}+2}\bigr ). \end{aligned}$$

(5)

Then we can paint the vertices from a corresponding independent set in one color. Now if we remove colored vertices from G, we get a 3-graph with less than

$$\begin{aligned} B'r^{\frac{b}{b-1}+2} - \frac{1}{2} \cdot 2^{k}cr^2\cdot \alpha (G[V_k]) = B'(r-1)^{\frac{b}{b-1}+2} \end{aligned}$$

edges that is not $(r-1)$-colorable. That is in contradiction to the minimality of r. So we can assume that for every k the contrary of (5) holds, so, by upper bound from (1), we have

$$\begin{aligned} |V_k| < B\left( \frac{B'}{2^{k-1}cr^2}\bigl (r^{\frac{b}{b-1} + 2} - (r-1)^{\frac{b}{b-1}+2}\bigr ) \right) ^b. \end{aligned}$$

Using the fact that $x^q - (x-1)^q \le qx^{q-1}$ for $x,q \ge 1$, we get

$$\begin{aligned} |V_k| < B\left( \frac{B'}{2^{k-1}cr^2}\cdot \biggl (\frac{b}{b-1}+2\biggr ) \cdot r^{\frac{b}{b-1} + 1} \right) ^b = C \left( \frac{r^{\frac{b}{b-1}+1}}{2^{k-1}r^2}\right) ^b = \frac{1}{2^{b(k-1)}}\cdot C r^{\frac{b}{b-1}}, \end{aligned}$$

where

$$\begin{aligned} C = \frac{B{B'}^{b}\bigl (\frac{b}{b-1}+2\bigr )^b}{c^b}. \end{aligned}$$

Now let $g_k$ be the number of edges that pass through vertices of $V_k$. We have

$$\begin{aligned} |E(G[V_{big}])|{} & {} < \sum _{k \ge 0} g_k< \sum _{k \ge 0} 2^{k+1}cr^2 \cdot \frac{1}{2^{b(k-1)}}\cdot C r^{\frac{b}{b-1}} \\ {}{} & {} = cC2^{b+1}r^{\frac{b}{b-1}+2} \sum _{k\ge 0} 2^{(1-b)k} < \frac{cC2^{b+1}}{1-2^{1-b}} \cdot r^{\frac{b}{b-1}+2}. \end{aligned}$$

Combining this with (4) and substituting C and c we get

$$\begin{aligned} \frac{B'}{2^{\frac{b}{b-1}+2}} < \frac{(1/12e)B{B'}^b\bigl (\frac{b}{b-1}+2\bigr )^b 2^{b+1}}{(1/12e)^b(1-2^{1-b})} \end{aligned}$$

or

$$\begin{aligned} {B'}^{b-1} > \frac{(1-2^{1-b})}{2^{\frac{b}{b-1}+b+3} (12e)^{b-1}B\bigl (\frac{b}{b-1}+2\bigr )^b}, \end{aligned}$$

which is a contradiction.

2.2 Proof of Proposition 1

A direct application of Theorem 5 gives the upper bound $r^{9+o(1)}$. To obtain a better upper bound, let us count the number of edges in the example constructed in [8]. The construction is as follows: let $G_q$ be a generalized quadrangle, i.e. a $(q + 1)$-regular $(q + 1)$-uniform simple $C_3$-free hypergraph on $q^3 + q^2 + q + 1$ vertices. It is known that such hypergraph exists whenever q is a power of a prime number. Each edge of $G_q$ is then replaced by a random 3-graph $F_q$ which is built as follows: vertices of $F_q$ are randomly split into a set $V = \{v_{ij} \ |\ 1 \le i,j\le \tau \}$ and $2\tau$ sets $S_1, \dots , S_{\tau }, T_1, \dots , T_{\tau }$ of roughly the same size, where $\tau = \lfloor 8 \sqrt{\log q} \rfloor$. Edges of $F_q$ are all sets of the form $\{v_{ij}, a, b\}$, where $a \in S_i$, $b \in T_j$.

The result of the construction is a hypergraph on $cq^3$ vertices. With a high probability its independence number does not exceed

$$\begin{aligned} \frac{2\tau n}{q}=16q^2\sqrt{\log q}, \end{aligned}$$

i.e. its chromatic number is at least $cq/\sqrt{\log q}$.

At the same time, this hypergraph has

$$\begin{aligned} cq^3\tau ^2\left( \frac{q+1-\tau ^2}{2\tau }\right) ^2=cq^5 \end{aligned}$$

edges. Substituting $cq/\sqrt{\log q}=r$, we get

$$\begin{aligned} m_{C_3}(r) \le cr^5(\log r)^{5/2}. \end{aligned}$$

2.3 Proof of Theorem 6

2.3.1 The Upper Bound

For any $t\ge 1$ there is a H-free 3-graph $G_t$ with $|V(G_t)| = t^{At}$ and $\alpha (G_t) < t$. Then

$$\begin{aligned} \chi (G_t) \ge \frac{|V(G_t)|}{\alpha (G_t)} > \frac{t^{At}}{t} = t^{At-1}, \end{aligned}$$

so one may put $r = t^{At-1}$. First we prove the desired bound only for such r. Hence

$$\begin{aligned} \ln r = (At-1)\ln t, \end{aligned}$$

so

$$\begin{aligned} \frac{1}{A} \frac{\ln r}{\ln \ln r} = \frac{1}{A} \frac{(At-1)\ln t}{\ln ((At-1)\ln t)} = (1+o(1))t. \end{aligned}$$

Finally,

$$\begin{aligned} |E(G_t)| \le |V(G_t)|^3 = t^{3At} = r^3t^3 = (1+o(1)) \left( \frac{r \ln r}{A \ln \ln r}\right) ^3. \end{aligned}$$

Then, consider any r between $(t-1)^{A(t-1)-1}$ and $t^{At-1}$. Let $A'$ be such that $r = t^{A't-1}$. For a large enough t one has

$$\begin{aligned} t^{At/2-1} < (t-1)^{A(t-1)-1}, \end{aligned}$$

so $A> A' > A/2$. Consider any induced subgraph $G'_t$ of $G_t$ on $t^{A't}$ vertices. It is H-free and has the independence number smaller than t, so the chromatic number of $G'_t$ is at least r. Then one may repeat the previous argument to show

$$\begin{aligned} |E(G'_t)|\le & {} |V(G'_t)|^3 = t^{3A't} = r^3t^3 = (1+o(1)) \left( \frac{r \ln r}{A' \ln \ln r}\right) ^3 \\< & {} (1+o(1)) \left( \frac{2 r \ln r}{A \ln \ln r}\right) ^3. \end{aligned}$$

2.3.2 The Lower Bound

Essentially, this proof follows along the proof of the lower bound in Theorem 5. First, we show that for each $\varepsilon \in (0, 1 / b)$ there is some constant $C_\varepsilon > 0$ such that

$$\begin{aligned} m_H(r) \ge C_\varepsilon r^3 \ln ^{1/b-\varepsilon }r. \end{aligned}$$

(6)

Suppose that G is an H-free 3-graph with $|E(G)| < C_\varepsilon r^3\ln ^{1/b-\varepsilon }r$ that is not r-colorable. Here $C_\varepsilon$ is some constant which depends only on $\varepsilon$ and which will be chosen later. Let such r be minimal possible.

By Lemma 3 we have that $G[V_{big}]$ should not be $\lfloor r/2\rfloor$-colorable.

Let us look at the value of $\alpha (G[V_{big}])$. If

$$\begin{aligned} \alpha (G[V_{big}]) \ge \frac{2C_\varepsilon }{cr^2}\bigl ( r^{3}\ln ^{1/b-\varepsilon }r - (r-1)^3\ln ^{1/b-\varepsilon }(r-1) \bigr ), \end{aligned}$$

(7)

then we can paint vertices of the biggest independent set of $G[V_{big}]$ in one color. By removing colored vertices from G we get H-free 3-graph that is not $(r-1)$-colorable but has less than

$$\begin{aligned} C_\varepsilon r^3\ln ^{1/b-\varepsilon }r - \frac{cr^2}{2} \cdot \frac{2C_\varepsilon }{cr^2}\bigl ( r^{3}\ln ^{1/b-\varepsilon }r - (r-1)^3\ln ^{1/b-\varepsilon }(r-1) \bigr ) = C_\varepsilon (r-1)^3\ln ^{1/b-\varepsilon }(r-1) \end{aligned}$$

edges, which contradicts the minimality of r.

So the contrary to (7) must hold. Since $r^3\ln ^a r - (r-1)^3\ln ^a(r-1) < 4r^2\ln ^a r$ for all $r\ge 1$ and $a \ge 0$, we have

$$\begin{aligned} \alpha (G[V_{big}]) < \frac{8C_\varepsilon }{c} \cdot \ln ^{1/b-\varepsilon }r. \end{aligned}$$

Due to the bound (2),

$$\begin{aligned} \left| V_{big} \right| < \left( \frac{8C_\varepsilon }{c} \cdot \ln ^{1/b-\varepsilon }r\right) ^{B((8C_\varepsilon \ln ^{1/b-\varepsilon }r)/ c)^b}. \end{aligned}$$

We take logarithm of the right-hand side and see that when $C_\varepsilon \le c/8$

$$\begin{aligned} B\left( \frac{8C_\varepsilon }{c} \ln ^{1/b-\varepsilon } r\right) ^b \cdot \left( \ln \frac{8C_\varepsilon }{c} + \ln \ln ^{1/b-\varepsilon }r\right){} & {} \le B\left( \frac{8C_\varepsilon }{c}\right) ^b \ln ^{1-b\varepsilon } r \cdot \ln \ln ^{1/b-\varepsilon }r\\ {}{} & {} < B\left( \frac{8C_\varepsilon }{c}\right) ^b\ln ^{1-b\varepsilon } r \cdot \ln \ln ^{1/b}r. \end{aligned}$$

There exists $x_\varepsilon > 0$ such that the function $\ln ^{b\varepsilon } x / \ln \ln ^{1/b} x$ is positive and strictly increases when $x \ge x_\varepsilon$. Then for any $C_\varepsilon$ such that

$$\begin{aligned} \left( \frac{8C_\varepsilon }{c} \cdot \ln ^{1/b-\varepsilon }x\right) ^{B((8C_\varepsilon \ln ^{1/b-\varepsilon }x)/ c)^b}< x \text { for any } x< x_\varepsilon \text { and } C_\varepsilon ^b < \frac{1}{B} \cdot \left( \frac{c}{8}\right) ^b \frac{\ln ^{b\varepsilon } x_\varepsilon }{\ln \ln ^{1/b} x_\varepsilon }, \end{aligned}$$

we either have $|V_{big}| < r$ if $r < x_\varepsilon$, or we have

$$\begin{aligned} \ln |V_{big}|< BC_\varepsilon ^b\left( \frac{8}{c}\right) ^b \ln ^{1-b\varepsilon } r \cdot \ln \ln ^{1/b} r < \ln ^{1-b\varepsilon } r \cdot \ln ^{b\varepsilon } r = \ln r \end{aligned}$$

if $r \ge x_\varepsilon$, which also means that $|V_{big}| < r$. So vertices of $V_{big}$ can be colored in $\lfloor r/2 \rfloor$ colors with no more than two vertices of same color. It means that there is a proper coloring of G in r colors. Obtained contradiction finishes the proof of (6).

It remains to construct some function g such that $g(r) \rightarrow 0$ as $r \rightarrow \infty$ and

$$\begin{aligned} m_H(r) \ge r^3 \ln ^{1/b-g(r)} r. \end{aligned}$$

For each n there is some $r_n$ such that for $r \ge r_n$

$$\begin{aligned} C_{1/n} \ln ^{1/n} r > 1, \end{aligned}$$

which means that

$$\begin{aligned} m_H(r) \ge r^3 \ln ^{1/b-2/n} r. \end{aligned}$$

Finally, for $r \ge r_1$ let

$$\begin{aligned} g(r) = \frac{2}{\max \limits _{\begin{array}{c} n \le r\\ r_n \le r \end{array}} n}. \end{aligned}$$

Clearly, g(r) is non-increasing and can be arbitrary small, so $g(r) \xrightarrow [r\rightarrow \infty ]{}0$.

2.4 Proof of Proposition 2

Suppose the contrary, i. e. there is an H-free 3-graph G with the chromatic number more than r and the number of edges at most $Cr^4$. Let such r be minimal possible.

Let us try to color $G[V_{big}]$ in $r'=\lfloor r/2 \rfloor$ colors greedily with respect to an arbitrary linear order $\pi$ on the vertex set. If it is impossible then there is a vertex v such that it is the last vertex in edges $f_1,\dots ,f_{r'}$ such that set $f_i {\setminus } \{v\}$ has color i. Then an arbitrary choice of one vertex from each set $f_i \setminus \{v\}$ leads to an independent set I of size $r'$, since G is H-free.

By removing I from G we get some 3-graph that is not $(r-1)$-colorable, so it must have more than $C(r-1)^4$ edges. So we removed less than

$$\begin{aligned} Cr^4 - C(r-1)^4 < C\cdot 4r^3 = \frac{er^3}{72} \end{aligned}$$

edges. But the sum of degrees of vertices in I (in the initial graph G) is at least $r'\cdot cr^2$, so it intersects at least

$$\begin{aligned} \frac{cr^2r'}{2} \ge \frac{cr^3}{6} = \frac{er^3}{72} \end{aligned}$$

edges. That is a contradiction.

2.5 Proof of Theorem 7

Let $C_2 > 0$. Consider an H-free 3-graph G with

$$\begin{aligned} |E(G)| \le \frac{C_2r^{\frac{2b}{b-1}}}{(\ln \ln r)^{1+\frac{2b}{b-1}}}. \end{aligned}$$

(8)

We will show that it is r-colorable if r is sufficiently big. Let $V_0 = V(G)$ and

$$\begin{aligned} V_i = \left\{ v \in V_{i-1}: \deg _{G[V_{i-1}]} v \ge \frac{1}{C_3}\left( \frac{r}{\ln \ln r} \right) ^2 \right\} , \quad i \ge 1, \end{aligned}$$

where $C_3 > 0$ is chosen to satisfy the inequality

$$\begin{aligned} \frac{1}{C_3} \left( \frac{r}{\ln \ln r} \right) ^2 \le \frac{1}{3e} \left[ \frac{r}{\log _b\ln r} \right] ^2 \end{aligned}$$

for, let’s say, $r > 100$, so, by Lemma 2, every 3-graph $G[V_i {\setminus } V_{i+1}]$ for $i\ge 0$ has a proper coloring in $\bigl [\frac{r}{\log _b\ln r}\bigr ]$ colors. Clearly, $C_3$ can be chosen independently of r.

Now we are going to estimate sizes of $E(G[V_i])$ and $V_{i+1}$ from above for each $i \ge 0$. As $V_{i+1} \subset V_i$, we have

$$\begin{aligned} \left| E(G[V_i]) \right| \ge \frac{1}{3} \sum _{v \in V_{i+1}} \deg _{G[V_i]} v \ge \frac{1}{3} \cdot \left| V_{i+1} \right| \cdot \frac{1}{C_3} \left( \frac{r}{\ln \ln r} \right) ^2, \end{aligned}$$

so

$$\begin{aligned} \left| V_{i+1} \right| \le 3C_3 \left( \frac{\ln \ln r}{r} \right) ^2 \left| E(G[V_i]) \right| . \end{aligned}$$

(9)

Also, we have bound (8) for $\left| E(G[V_0]) \right| = |E(G)|$, and, by (3),

$$\begin{aligned} \left| E(G[V_i]) \right| \le C_1 (\left| V_i \right| )^b \ \text {for }i \ge 1. \end{aligned}$$

(10)

Combining (10) and (9) and using induction on i, we obtain for $i\ge 0$

$$\begin{aligned}{} & {} \left| E(G[V_{i}]) \right| \le C_1^{\frac{b^i-1}{b-1}} C_2^{b^i} (3C_3)^{\frac{b^{i+1}-b}{b-1}} \cdot \frac{r^{\frac{2b}{b-1}}}{(\ln \ln r)^{b^{i}+\frac{2b}{b-1}}}, \\{} & {} \left| V_{i+1} \right| \le C_1^{\frac{b^i-1}{b-1}} C_2^{b^i} (3C_3)^{\frac{b^{i+1}-1}{b-1}} \cdot \frac{r^\frac{2}{b-1}}{(\ln \ln r)^{b^i + \frac{2}{b-1}}}. \end{aligned}$$

The last inequality implies that $V_N$ is empty for N and r large enough. Namely, let $C = C_1^{\frac{1}{b-1}}C_2(3C_3)^{\frac{b}{b-1}}$. Then for $r > r_0(C_1,C_3)$ from last inequality we get

$$\begin{aligned} \left| V_{i+1} \right| \le \frac{r^{\frac{2}{b-1}}}{C_1^{\frac{1}{b-1}} (3C_3)^{\frac{1}{b-1}} \cdot (\ln \ln r)^{\frac{2}{b-1}}} \bigg / \left( \frac{\ln \ln r}{C} \right) ^{b^{i}} \le r^{\frac{2}{b-1}} \bigg / \left( \frac{\ln \ln r}{C} \right) ^{b^{i}}. \end{aligned}$$

(11)

So for $N > \log _b \ln r$

$$\begin{aligned} |V_{N+1}| \le r^{\frac{2}{b-1}} \bigg / \left( \frac{\ln \ln r}{C} \right) ^{\ln r} = \left( \frac{C e^{\frac{2}{b-1}}}{\ln \ln r} \right) ^{\ln r}, \end{aligned}$$

which is less than 1 when $\ln \ln r > Ce^{2/(b-1)}$, i.e.

$$\begin{aligned} r > e^{e^{Ce^{2/(b-1)}}}. \end{aligned}$$

We have divided V into at most $\log _b \ln r$ sets of form $V_i\setminus V_{i+1}$, and each 3-graph $G[V_i\setminus V_{i+1}]$ has a proper coloring with at most $\frac{r}{\log _b\ln r}$ colors, so G is r-colorable. Therefore, we can put $R = \max \left( 100, r_0, e^{e^{Ce^{2/(b-1)}}} \right)$.

2.6 Proof of Proposition 3

Suppose that a 3-graph G has more than $(t-3)\left( {\begin{array}{c}n\\ 2\end{array}}\right)$ edges. We will show that some subgraph of G is isomorphic to H.

Let us repeat the following procedure: if there is an edge $g = \{v_1,v_2,v_3\}$ such that the pair $(v_1,v_2)$ is contained in at most $t-3$ edges (including g), remove g. There will be removed at most $(t-3)\left( {\begin{array}{c}n\\ 2\end{array}}\right)$ edges, and after that each pair of vertices will be joined by either 0 or at least $t-2$ edges. Denote resulting 3-graph by $G'$.

Let $H_i$ be a subgraph of H induced by edges $h_1,\dots ,h_i$. We have removed at most $(t-3)\left( {\begin{array}{c}n\\ 2\end{array}}\right)$ edges from G, so $\left| E(G') \right| \ge 1$. A subgraph of $G'$ induced by one edge is isomorphic to $H_1$. Now we will show that if there is a subgraph $H'_{i - 1} \subset G'$ isomorphic to $H_{i-1}$, then there is a subgraph isomorphic to $H_{i}$. All we have to do is find an edge which can be put in correspondence to $h_i$. $H_{i-1}$ has an edge $h_{j(i)} \supset \{a_i, b_i\}$. Consider $v,u \in V(H'_{i - 1})$ that correspond to $a_i$ and $b_i$. There is an edge in $H'_{i - 1}$ that corresponds to $h_{j(i)}$, so v and u are joined by at least $t-2$ edges in $G'$. We have $\left| V(H'_{i - 1}) \right| - 2 = \left| V(H_{i-1}) \right| - 2 = i - 1 < t - 2$, so there exists $e \in E(G')$ such that $e = \{v, u, w\}$ and $w \notin V(H'_{i - 1})$. This edge e can be put in correspondence to $h_i$.

So there is a subgraph of $G'$ that is isomorphic to $H_{t-2} = H$ and, since $G'\subset G$, it is a subgraph of G.

3 Discussion

Questions from [1].

Conjecture 1

There exists a simple 3-graph H for which

$$\begin{aligned} m_H(r) = r^{3+o(1)}. \end{aligned}$$

Problem 1

Characterize all 3-graphs H such that

$$\begin{aligned} m_H(r) = r^{3+o(1)}. \end{aligned}$$

Regularity Issues. It is known [3] that sequence $\frac{m(r)}{r^3}$ has a positive limit. The same statement for quantity $m_H(r)$ is of interest (H is assumed to be fixed). In particular, it is unknown even for the case of simple 3-graphs, i.e. when $H = \{\{1,2,3\}, \{1,2,4\}\}$, for which the asymptotics is known:

$$\begin{aligned} m_H(r) = \Theta (r^4\log ^2 r). \end{aligned}$$