|
|||||
FRAMES NO FRAMES | |||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Field Summary | |
---|---|
def |
EMPTY_BYTE_ARRAY
|
def |
buffer
|
Charset |
charset
|
Charset |
defaultCharset
|
boolean |
enforce8Bit
|
File |
file
|
Constructor Summary | |
CharsetToolkit(File file)
Constructor of the CharsetToolkit utility class.
|
Method Summary | |
---|---|
static def |
getAvailableCharsets()
Retrieves all the available Charset s on the platform,
among which the default charset .
|
Charset |
getCharset()
|
Charset |
getDefaultCharset()
Retrieves the default Charset |
static Charset |
getDefaultSystemCharset()
Retrieve the default charset of the system. |
boolean |
getEnforce8Bit()
Gets the enforce8Bit flag, in case we do not want to ever get a US-ASCII encoding. |
BufferedReader |
getReader()
Gets a BufferedReader (indeed a LineNumberReader ) from the File
specified in the constructor of CharsetToolkit using the charset discovered by the
method guessEncoding() .
|
Charset |
guessEncoding()
Guess the encoding of the provided buffer. |
boolean |
hasUTF16BEBom()
Has a Byte Order Marker for UTF-16 Big Endian (utf-16 and ucs-2). |
boolean |
hasUTF16LEBom()
Has a Byte Order Marker for UTF-16 Low Endian (ucs-2le, ucs-4le, and ucs-16le). |
boolean |
hasUTF8Bom()
Has a Byte Order Marker for UTF-8 (Used by Microsoft's Notepad and other editors). |
static boolean |
isContinuationChar(byte b)
If the byte has the form 10xxxxx, then it's a continuation byte of a multiple byte character; |
static boolean |
isFiveBytesSequence(byte b)
If the byte has the form 11110xx, then it's the first byte of a five-bytes sequence character. |
static boolean |
isFourBytesSequence(byte b)
If the byte has the form 11110xx, then it's the first byte of a four-bytes sequence character. |
static boolean |
isSixBytesSequence(byte b)
If the byte has the form 1110xxx, then it's the first byte of a six-bytes sequence character. |
static boolean |
isThreeBytesSequence(byte b)
If the byte has the form 1110xxx, then it's the first byte of a three-bytes sequence character. |
static boolean |
isTwoBytesSequence(byte b)
If the byte has the form 110xxxx, then it's the first byte of a two-bytes sequence character. |
void |
setDefaultCharset(Charset defaultCharset)
Defines the default Charset used in case the buffer represents
an 8-bit Charset .
|
void |
setEnforce8Bit(boolean enforce)
If US-ASCII is recognized, enforce to return the default encoding, rather than US-ASCII. |
Constructor Detail |
---|
public CharsetToolkit(File file)
CharsetToolkit
utility class.
Method Detail |
---|
public static def getAvailableCharsets()
Charset
s on the platform,
among which the default charset
.
Charset
s.
public Charset getCharset()
public Charset getDefaultCharset()
public static Charset getDefaultSystemCharset()
Charset
.
public boolean getEnforce8Bit()
public BufferedReader getReader()
BufferedReader
(indeed a LineNumberReader
) from the File
specified in the constructor of CharsetToolkit
using the charset discovered by the
method guessEncoding()
.
BufferedReader
Charset guessEncoding()
Guess the encoding of the provided buffer.
If Byte Order Markers are encountered at the beginning of the buffer, we immidiately return the charset implied by this BOM. Otherwise, the file would not be a human readable text file.If there is no BOM, this method tries to discern whether the file is UTF-8 or not. If it is not UTF-8, we assume the encoding is the default system encoding (of course, it might be any 8-bit charset, but usually, an 8-bit charset is the default one).
It is possible to discern UTF-8 thanks to the pattern of characters with a multi-byte sequence.
UCS-4 range (hex.) UTF-8 octet sequence (binary) 0000 0000-0000 007F 0xxxxxxx 0000 0080-0000 07FF 110xxxxx 10xxxxxx 0000 0800-0000 FFFF 1110xxxx 10xxxxxx 10xxxxxx 0001 0000-001F FFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx 0020 0000-03FF FFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 0400 0000-7FFF FFFF 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
With UTF-8, 0xFE and 0xFF never appear.
public boolean hasUTF16BEBom()
public boolean hasUTF16LEBom()
public boolean hasUTF8Bom()
static boolean isContinuationChar(byte b)
static boolean isFiveBytesSequence(byte b)
static boolean isFourBytesSequence(byte b)
static boolean isSixBytesSequence(byte b)
static boolean isThreeBytesSequence(byte b)
static boolean isTwoBytesSequence(byte b)
public void setDefaultCharset(Charset defaultCharset)
Charset
used in case the buffer represents
an 8-bit Charset
.
Charset
to be returned by guessEncoding()
Charset
is encountered.
public void setEnforce8Bit(boolean enforce)
charset
rather than US-ASCII.