在文件比較大時(shí),我們不能一次將整個(gè)文件讀取到內(nèi)存中,需要按行讀并按行處理,但是這樣帶來(lái)的問(wèn)題在于將讀取文件的代碼與處理數(shù)據(jù)的代碼混在了一起:
try (BufferedReader in = new BufferedReader(new FileReader(file))) {
String line;
while ((line = in.readLine()) != null) {
doProcessLine(line);
if (!doNext(line)) {//省略
break;
}
}
}
可以使用Guava將它分開,例如:
CharStreams.readLine(reader, new LineProcessor<T>() {
public boolean processLine(String line) throws IOException {
if (!doNext(line)) {
return false;//不需要再繼續(xù)讀
}
...
}
public T getResult() {
return xxx;
}
});
這樣代碼比較繁瑣,不自然。
更自然的方式:
for (String line : IOStreams.lines(reader)) {
if (!doNext(line)) {
break;//停止讀,因?yàn)槭莑azy的,當(dāng)前行處理結(jié)束前,下一行不會(huì)被讀入到內(nèi)存
}
...
}
實(shí)現(xiàn)方式:
先定義兩個(gè)基礎(chǔ)接口CloseableIterable和CloseableIterator:
import java.io.Closeable;
/**
* 可被關(guān)閉的{@link Iterable}對(duì)象,可用于基于某種資源(如ResultSet)的迭代。
* <p>
* <pre>
* {@code
* try (CloseableIterable<E> iter = xxx) {
* for (E elem : iter) {
* ……
* }
* }
* }
* </pre>
*
* @author gaohang on 15/9/21.
*/
public interface CloseableIterable<E> extends Iterable<E>, Closeable {
/**
* 關(guān)閉資源
*/
void close();
}
import java.io.Closeable;
import java.util.Iterator;
/**
* @author gaohang on 15/9/21.
* @see CloseableIterable
*/
public interface CloseableIterator<E> extends Iterator<E>, Closeable {
void close();
}
最后實(shí)現(xiàn)IOStreams類:
import cn.yxffcode.freetookit.collection.ImmutableIterator;
import cn.yxffcode.freetookit.lang.CloseableIterable;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.Iterator;
import static com.google.common.base.Preconditions.checkNotNull;
/**
* 輸入輸出流相關(guān)的工具
*
* @author gaohang on 15/10/23.
*/
public final class IOStreams {
private IOStreams() {
}
/**
* 按行讀取
* <p>
* 使用lazy的讀,只有在返回的Iterable對(duì)象上迭代一次才會(huì)讀一行
* <p>
* 當(dāng)讀取的文件比較大時(shí),不合適一次將事個(gè)文件的行讀入內(nèi)存,需要逐行讀并處理。在使用guava的CharStreams時(shí)比較繁鎖,例如:
* <pre>
* CharStreams.readLine(reader, new LineProcessor<T>() {
* public boolean processLine(String line) throws IOException {
* if (line ....) {
* return false;//不需要再繼續(xù)讀
* }
* ...
* }
* public T getResult() {
* return xxx;
* }
* });
* </pre>
* 換成此方法,代碼如下:
* <pre>
* for (String line : IOStreams.lines(reader)) {
* if (line...) {
* break;//停止讀,因?yàn)槭莑azy的,當(dāng)前行處理結(jié)束前,下一行不會(huì)被讀入到內(nèi)存
* }
* ...
* }
* </pre>
*/
public static Iterable<String> lines(final BufferedReader reader) {
checkNotNull(reader);
return new Iterable<String>() {
@Override public Iterator<String> iterator() {
return new ImmutableIterator<String>() {
private String line;
@Override public boolean hasNext() {
try {
return (line = reader.readLine()) != null;
} catch (IOException e) {
throw new IOReaderException(e);
}
}
@Override public String next() {
return line;
}
};
}
};
}
public static CloseableIterable<String> lines(final File src) {
return new CloseableIterable<String>() {
private BufferedReader in;
@Override public void close() {
if (in != null) {
try {
in.close();
} catch (IOException e) {
throw new IOReaderException(e);
}
}
}
@Override public Iterator<String> iterator() {
try {
in = new BufferedReader(new FileReader(src));
return lines(in).iterator();
} catch (FileNotFoundException e) {
throw new IOReaderException(e);
}
}
};
}
public static BufferedReader toBufferedReader(InputStream in) {
return new BufferedReader(new InputStreamReader(in));
}
public static BufferedReader openClasspath(String classpath) {
return toBufferedReader(IOStreams.class.getResourceAsStream(classpath));
}
}