C++ Standard Library <codecvt>

<codecvt>It is a header file in the C++ standard library that provides tools for character conversion. This header file mainly includesstd::codecvtclass templates and their specializations, supporting conversions between character encodings, such as from UTF-8 to UTF-16, or from wide characters (wchar_t) to narrow character (char), etc.std::codecvtThe class is usually associated withstd::wstring_convertclass together to achieve character encoding conversion.

Syntax

codecvtThe main classes and functions in the namespace are as follows:

  • codecvt_base: defines the state type and error handling methods for encoding conversion.
  • codecvt_byname: a template class used to create converters for specific encodings.
  • codecvt_utf8、codecvt_utf16: Converter class for specific encodings.

Basic Syntax

#include <codecvt>
#include <locale>
#include <string>

std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
std::wstring wide_string = converter.from_bytes("Hello, World!");
std::string narrow_string = converter.to_bytes(L"你好，世界！");

Example

Example 1: Conversion from UTF-8 to UTF-16

In this example, we will demonstrate how to usecodecvtto convert a UTF-8 encoded string to a UTF-16 encoded wide string.

Example

#include <iostream>
#include <codecvt>
#include <locale>
#include <string>

int main() {
    // Create a UTF-8 to UTF-16 converter
    std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;

    // Original UTF-8 string
    std::string narrow_string = "Hello, World!";

    // Convert to UTF-16 wide string
    std::wstring wide_string = converter.from_bytes(narrow_string);

    // Output wide string
    std::wcout << L"Wide string: " << wide_string << std::endl;

    // Convert the wide string back to a UTF-8 string
    std::string converted_string = converter.to_bytes(wide_string);

    // Output the converted string
    std::cout << "Converted string: " << converted_string << std::endl;

    return 0;
}

Output result:

Wide string: Hello, World!
Converted string: Hello, World!

Example 2: Encoding conversion using codecvt_byname

In this example, we will demonstrate how to usecodecvt_bynameclass to create a name-based encoding converter and use it for conversion.

Example

#include <iostream>
#include <codecvt>
#include <locale>
#include <string>

int main() {
    // Create a name-based converter; here "zh_CN.UTF-8" represents the UTF-8 encoding for Simplified Chinese
    std::wstring_convert<std::codecvt_byname<wchar_t>> converter("zh_CN.UTF-8");

    // Original UTF-8 string
    std::string narrow_string = "Hello, world!";

    // Convert to wide string
    std::wstring wide_string = converter.from_bytes(narrow_string);

    // Output wide string
    std::wcout << L"Wide string: " << wide_string << std::endl;

    // Convert the wide string back to a UTF-8 string
    std::string converted_string = converter.to_bytes(wide_string);

    // Output the converted string
    std::cout << "Converted string: " << converted_string << std::endl;

    return 0;
}

Output result:

Wide string: 你好，世界！
Converted string: 你好，世界！

std::codecvtClass template specialization

std::codecvtThere are multiple specialization versions for different character encoding conversions:

  • std::codecvt_utf8<wchar_t>: wide character (wchar_t) and UTF-8.
  • std::codecvt_utf8_utf16<char16_t>: Conversion between UTF-8 and UTF-16.
  • std::codecvt_utf8<char32_t>: Conversion between UTF-8 and UTF-32.

std::wstring_convertclass template

std::wstring_convertThe class template is a helper class used to manage the lifecycle and exception handling of character encoding conversion:

  • to_bytes: converts wide characters or strings in other encodings to narrow characters (byte sequences).
  • from_bytes: converts narrow characters (byte sequences) to wide characters or strings in other encodings.

Notes

  • In the C++17 standardstd::codecvthas been deprecated, and it is recommended to use alternative solutions (such as the ICU library) for character encoding conversion in the future.
  • For cross-platform applications, special care should be taken when handling character encoding to ensure consistent behavior across all platforms.

Summary

<codecvt>It provides a powerful set of tools for conversion between different character encodings, especially among UTF-8, UTF-16, and wide characters. Although deprecated in C++17, it remains a useful tool when handling character encoding conversion. Understanding and mastering the use of these tools can help you write more flexible and internationalized applications. If you have specific usage requirements or questions, you can discuss them further.

other extensions