Frugal

Frugal is a high-performance Thrift codec, based on JIT that does not rely on the Go codec based on generated Go source code. In most scenarios, it outperforms the Go codec.

Github Project Page: https://github.com/cloudwego/frugal

Usage

Note:

Frugal can be used independently on either server side or client side.
1. The transmitted data are always encoded according to the standard thrift protocol
If Frugal is enabled on the server side, make sure that Framed (or TTHeaderFramed) is enabled on the client side.
If slim template is used, PayloadCodec must be specified to enable frugal.

Kitex Integretion

Scaffold Generation

The Kitex Command Line tool has built-in frugal integration.

Command Line Parameters

Frugal Tag: -thrift frugal_tag

Generate Go structs with frugal tags, for example:

type Request struct {
    Message string `thrift:"message,1" frugal:"1,default,string" json:"message"`
}

Note:

Frugal relies on these tags. For example, set and list are both mapped to slice in golang, which can only be distinguished by frugal tag;
Without frugal tags, kitex will automatically fallback to the default Go codec (provided not using slim template);
If you don’t want to generate frugal tags, use -thrift frugal_tag=false.

Kitex >= v0.5.0 defaults to have this parameter specified; in older versions, you need to manually specify this parameter and execute the kitex command, for example:

kitex -thrift frugal_tag -service service_name idl/api.thrift

Pretouch: -frugal-pretouch

Call frugal.Pretouch method in init() to preprocess (JIT compile) all request/response types and reduce the time required for the first request.

Frugal defaults to calling the JIT Compiler before the first encoding/decoding on a specific type, which may cause the first request to take longer.

Example:

kitex -frugal-pretouch -service service_name idl/api.thrift

Slim Template: -thrift template=slim

Please upgrade thriftgo to v0.3.0 (or above); the structs generated by old versions have problems with the optional fields.

Do not generate the traditional Thrift Go codec code (which implements the interface thrift.TProtocol) to reduce the amount of source code and improve IDE loading and compiling speed.

Here’s the example code generated with slim template.

Frugal uses JIT to generate codec code and does not rely on the generated Go codec.

Example:

kitex -thrift frugal_tag,template=slim -service service_name idl/api.thrift

Note: Enabling Slim will result in no fallback and reporting an error when frugal is not supported (such as on ARM architecture, or inability to obtain the size of the thrift payload from the request header).

Example Usage

It is recommended to use the latest version of Kitex (>= v0.5.0) and thriftgo (>= v0.3.0).

Conservative version

kitex -thrift frugal_tag -service service_name idl/api.thrift

Explain:

New Kitex versions (>=v0.5.0) will generate frugal tags by default.
Do not use pretouch: commonly not all types are used in a project; you can try to enable it and observe whether it slows down the bootstrap of your service.
Do not use slim templates: In scenarios where frugal is not supported, fallback is applicable with the generated Go codec.

Radical version

kitex -thrift frugal_tag,template=slim -frugal-pretouch -service service_name idl/api.thrift

Explain:

Pretouch enabled: may cause the process to start slowly
Slim template enabled: In scenarios that do not support frugal, fallback to the generated Go codec is not applicable, and an error will be reported.

Kitex Server

Notes

Please ensure that the client has Framed (or TTHeaderFramed) enabled.

Enabling Framed ensures that the payload size is set in the header;
If the payload size is not available, currently Kitex Server can only fallback to the generated Go codec;
If it’s using slim template, fallback is not available and an error will be reported: “decode failed, codec msg type not match”.

server.Option

Related options for server initialization.

server.WithPayloadCodec

For enabling Frugal:

server.WithPayloadCodec(
    thrift.NewThriftCodecWithConfig(thrift.FrugalRead | thrift.FrugalWrite)
)

Note: If an error is reported (symbol not found), it indicates that the combination of the current kitex version + go version does not support frugal. For example, Go 1.21 + Kitex v0.7.1 (unsupported versions are blocked through conditional compile).

Example Code

Refer to: kitex-examples: frugal/server.go

package main

import (
    "context"

    "github.com/cloudwego/kitex-examples/kitex_gen/api"
    "github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
    "github.com/cloudwego/kitex/pkg/remote/codec/thrift"
    "github.com/cloudwego/kitex/server"
)

type EchoImpl struct{}

func (e EchoImpl) Echo(ctx context.Context, req *api.Request) (r *api.Response, err error) {
    return &api.Response{Message: req.Message}, nil
}

func frugalServer() {
    code := thrift.NewThriftCodecWithConfig(thrift.FrugalRead | thrift.FrugalWrite)
    svr := echo.NewServer(new(EchoImpl), server.WithPayloadCodec(code))
    err := svr.Run()
    if err != nil {
        panic(err)
    }
}

Kitex Client

client.Option

Related options for client initialization.

client.WithPayloadCodec

For enabling Frugal:

client.WithPayloadCodec(
    thrift.NewThriftCodecWithConfig(thrift.FrugalRead | thrift.FrugalWrite)
)

Note: If an error is reported (no symbol found), it indicates that the combination of the current kitex version + go version does not support frugal, for example, Go 1.21 + Kitex v0.7.1 (unsupported versions are blocked through conditional compile).

client.WithTransportProtocol

For enabling Framed: prepend a 4-byte (int32) length to indicate the size of thrift pure payload

client.WithTransportProtocol(transport.Framed)

Note:

If Framed is not set, there may be issues:
1. The server may not be able to decode with frugal without Payload Size (refer to: “Kitex Server -> Notes”);
2. The server won’t reply with a Framed payload, thus the Client may also not be able to decode with frugal (no Payload Size again);
If the target server does not support Framed, then don’t use it; the client can encode without it, but if the response (from the server) is not Framed (i.e. with preprended payload size), the client may not be able to decode with Frugal (so in this case, do not use slim template);
TTHeaderFramed is an alternative (it’s the BIT-OR result of TTHeader | Framed).

Example Code

Refer to: kitex-examples: frugal/client.go

package main

import (
    "context"

    "github.com/cloudwego/kitex-examples/kitex_gen/api"
    "github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
    "github.com/cloudwego/kitex/client"
    "github.com/cloudwego/kitex/pkg/klog"
    "github.com/cloudwego/kitex/pkg/remote/codec/thrift"
    "github.com/cloudwego/kitex/transport"
)

func frugalClient() {
    codec := thrift.NewThriftCodecWithConfig(thrift.FrugalRead | thrift.FrugalWrite)
    framed := client.WithTransportProtocol(transport.Framed)
    server := client.WithHostPorts("127.0.0.1:8888")
    cli := echo.MustNewClient("a.b.c", server, client.WithPayloadCodec(codec), framed)
    rsp, err := cli.Echo(context.Background(), &api.Request{Message: "Hello"})
    klog.Infof("resp: %v, err: %v", rsp, err)
}

Use Frugal Directly

In some scenarios (such as recording traffic), there’s a need to call frugal directly (without Kitex clients/servers).

Golang Struct

Frugal’s JIT compiler relies on Go structs with proper frugal tags.

Note:

A method may have multiple request arguments, so frugal.EncodeObject requires a Go struct encapsulating all these parameters in order. For example, the Struct EchoEchoArgs generated by Kitex encapsulates the Request.
Although the response has only one parameter, it should also be encapsulated in a struct. For example, EchoEchoResult encapsulates Response.
For details, please refer to the example code.

Use the kitex Command Line tool

Please refer to the section “Kitex Integration -> Code Generation” above. Note: kitex runs thriftgo to generate the structures.

Use thriftgo (>= v0.3.0)

Install thriftgo ( >= v0.3.0):

go install -v github.com/cloudwego/thriftgo@latest

Generate Go Structs based on a Thrift IDL:

thriftgo -r -o thrift -g go:frugal_tag,template=slim,package_prefix=github.com/example echo.thrift

Manually (Not Recommended)

Please refer to the struct generated by thriftgo (e.g. Request, Response).

Note:

Each field should have a frugal tag.
For optional fields, default values should be set in the `InitDefault()`` method.
Construct structs that encapsulate request/response arguments (e.g. EchoEchoArgs, EchoEchoResult)

Encoding

If you only want to use thrift codec (for example, to replace JSON), you can directly call frugal.EncodeObject(..).

If you want to generate a Thrift Payload that conforms to the Thrift Binary protocol encoding (which can be sent to the Thrift Server), the encoding result should include:

Thrift Magic Number: int16, the fixed value 0x8001
MessageType: int16, enumeration values, CALL = 1, REPLY = 2, Exception = 3, Oneway = 4
MethodName: length (int32) + name ([]byte)
Sequence ID：int32
Serialized request/response data

Among them, 1-4 can be achieved by calling the implementation in Kitex, and 5 can be achieved by calling frugal.EncodeObject(buf, nil, data).

Note: data should be a struct that encapsulates all request/response parameters in order (e.g. Struct EchoEchoArgs, EchoEchoResult generated by Kitex). Please refer to the example code for more details.

Decoding

According to Thrift Binary protocol encoding, decode result should contain:

MethodName
MessageType
Sequence ID
Unserialized request/response data

Among them, 1~3 can be decoded with the implementation in Kitex and 4 can be achieved by calling frugal.DecodeObject(buf, data).

Note: “data” should be a struct that encapsulates all request/response parameters in order (e.g. Struct EchoEchoArgs, EchoEchoResult generated by Kitex). Please refer to the example code for more details.

Example Code

Please refer to: kitex-examples: frugal/code/frugal.go

Precautions

Slim Template

Payload Size for Decoding: `Framed` recommended

When decoding, Kitex needs to obtain the Payload Size from the request header, to retrieve the complete Thrift PurePayload for decoding with frugal.

If the Slim template is enabled, and the Payload Size can NOT be obtained from the request, no fallback is available so Kitex could only report an error:

codec msg type not match with thriftCodec

Therefore, if the target server supports Framed, it is recommended to specify Framed (or TTHeaderFramed);

ARM support: not implemented yet

Since currently frugal does NOT support ARM architecture, projects that may be deployed to ARM machines should not use the slim template.

Mac M1/M2 users can use Rosetta to run frugal.
The slim template does not contain the generated Go codec (only JIT), so fallback is not available.

Performance Test Result

Traditional Thrift serializer and deserializer are based on generated Go code, which is no longer needed since we can use JIT compilation to dynamically generate machine code.

Thanks to JIT compilation, Frugal can generate better machine code than Go language compiler. In multi-core scenarios, Frugal’s performance could be as much as 5 times faster than traditional serializer and deserializer.

Below are the performance test results:

name                                 old time/op    new time/op     delta
MarshalAllSize_Parallel/small-16       78.8ns ± 0%     14.9ns ± 0%    -81.10%
MarshalAllSize_Parallel/medium-16      1.34µs ± 0%     0.32µs ± 0%    -76.32%
MarshalAllSize_Parallel/large-16       37.7µs ± 0%      9.4µs ± 0%    -75.02%
UnmarshalAllSize_Parallel/small-16      368ns ± 0%       30ns ± 0%    -91.90%
UnmarshalAllSize_Parallel/medium-16    11.9µs ± 0%      0.8µs ± 0%    -92.98%
UnmarshalAllSize_Parallel/large-16      233µs ± 0%       21µs ± 0%    -90.99%

name                                 old speed      new speed       delta
MarshalAllSize_Parallel/small-16     7.31GB/s ± 0%  38.65GB/s ± 0%   +428.84%
MarshalAllSize_Parallel/medium-16    12.9GB/s ± 0%   54.7GB/s ± 0%   +322.10%
MarshalAllSize_Parallel/large-16     11.7GB/s ± 0%   46.8GB/s ± 0%   +300.26%
UnmarshalAllSize_Parallel/small-16   1.56GB/s ± 0%  19.31GB/s ± 0%  +1134.41%
UnmarshalAllSize_Parallel/medium-16  1.46GB/s ± 0%  20.80GB/s ± 0%  +1324.55%
UnmarshalAllSize_Parallel/large-16   1.89GB/s ± 0%  20.98GB/s ± 0%  +1009.73%

name                                 old alloc/op   new alloc/op    delta
MarshalAllSize_Parallel/small-16         112B ± 0%         0B        -100.00%
MarshalAllSize_Parallel/medium-16        112B ± 0%         0B        -100.00%
MarshalAllSize_Parallel/large-16         779B ± 0%        57B ± 0%    -92.68%
UnmarshalAllSize_Parallel/small-16     1.31kB ± 0%     0.10kB ± 0%    -92.76%
UnmarshalAllSize_Parallel/medium-16      448B ± 0%      3022B ± 0%   +574.55%
UnmarshalAllSize_Parallel/large-16     1.13MB ± 0%     0.07MB ± 0%    -93.54%

name                                 old allocs/op  new allocs/op   delta
MarshalAllSize_Parallel/small-16         1.00 ± 0%       0.00        -100.00%
MarshalAllSize_Parallel/medium-16        1.00 ± 0%       0.00        -100.00%
MarshalAllSize_Parallel/large-16         1.00 ± 0%       0.00        -100.00%
UnmarshalAllSize_Parallel/small-16       6.00 ± 0%       1.00 ± 0%    -83.33%
UnmarshalAllSize_Parallel/medium-16      6.00 ± 0%      30.00 ± 0%   +400.00%
UnmarshalAllSize_Parallel/large-16      4.80k ± 0%      0.76k ± 0%    -84.10%

FAQ

How to avoid generating frugal tags?

The frugal tag generated by default has no impact on performance and is recommended to be reserved.

Run kitex with -thrift frugal_tag=false.

Note:

If frugal tags are not generated, you can not use frugal codec
1. In Golang, Thrift’s set and list are both mapped to slice, and can only be distinguished by frugal tags.
2. If Kitex detects that the request/response type does not contain a tag, it will NOT use frugal, and fallback to the standard Go codec.
If slim template is used, frugal tags must be generated.

Kitex Client Reporting “encode failed: codec msg type not match with thriftCodec”

Error message reported by client:

failed with error: remote or network error[remote]: encode failed, codec msg type not match with thriftCodec

Possible reasons:

Using the slim template WITHOUT specifying client.PayloadCodec;
Using the slim template WITHOUT generating frugal tags for Structs;

Kitex Server Reporting “decode failed, codec msg type not match with thriftCodec”

Error message reported by server (or in the client’s log):

decode failed, codec msg type not match with thriftCodec

Possible reasons:

Using the slim template WITHOUT specifying server.PayloadCodec;
Using the slim template WITHOUT generating frugal tags for Structs
The client is NOT sending Thrift payload with Framed or TTHeaderFramed

frugal: type mismatch: 11 expected, got 10

According to Thrift binary protocol, 11 is BINARY (or string), and 10 is an I64 type (not the field number in IDL).

According to the definition in Thrift IDL, a STRING is expected for the current field, but I64 is received, so frugal can not decode the payload.

Please make sure both the downstream and upstream are using the same IDL, and the generated source code is consistent with the IDL (could be the code is not regenerated with the updated IDL, or is not properly submitted to git).

Random stuff in the strings after decoding

With frugal <= v0.1.3, the NOCOPY mode is used by default for decoding strings (directly referencing the byte buffer provided to frugal.EncodeObject); but Kitex will reuse the byte buffer, resulting in the “value” of the string being modified.

New versions have NOCOPY mode disabled by default, which can be fixed after upgrading.

Error reported compiling Kitex projects: “undefined: thrift.FrugalRead”

Possible reasons:

Compiling with unsupported Go versions: please compile with go1.16 ~ go1.21;
The Kitex version does not support the Go version used: please upgrade to the latest Kitex
1. For example: Kitex v0.7.1 have frugal disabled when compiling with go1.21 (go1.21 was not supported by frugal when releasing these versions). Upgrading to Kitex >= v0.7.2 will fix it.

Optional fields are not filled with defaults with Slim template

Known issue: old versions of thriftgo do not generate InitDefault() with slim template.

Please upgrade to thriftgo >= v0.3.0 and regenerate the code with slim template.

frugal EncodeObject Reporting “unexpected EOF: 38 bytes short”

Since frugal.EncodeObject requires a byte buffer, Kitex calls frugal.EncodeSize(data) to calculate the required size for allocating the buffer, and then encodes the data.

If, between “calculating the buffer size” and “encoding the data”, another goroutine is modifying the object concurrently, the encoded result may exceed the given buffer, which may cause this error (or even panic).

This is not a bug in frugal. Avoid modifying the Request/Response passed to Kitex, including objects indirectly referenced in its fields, especially non-fixed-length types such as strings and slices.

frugal EncodeObject panic

It may be a known issue in the old versions. It’s recommended to upgrade to the latest version (>= v0.1.8):

go get github.com/cloudwego/frugal@latest

If the problem still exists, please confirm that the object being encoded is not being read and written concurrently (including objects indirectly referenced).

For example, if a string variable is being set to an empty string, a read on it may get an invalid string object (StringHeader.Data = nil && StringHeader.Len > 0), which will cause a “nil pointer error” panic when encoding.

Feedback

Was this page helpful?

Please tell us how we can improve.

Last modified March 12, 2024 : doc: sync User guide to Kitex multi-service (#1024) (1da567f)

Frugal

Usage

Kitex Integretion

Scaffold Generation

Command Line Parameters

Frugal Tag: -thrift frugal_tag

Pretouch: -frugal-pretouch

Slim Template: -thrift template=slim

Example Usage

Conservative version

Radical version

Kitex Server

Notes

server.Option

server.WithPayloadCodec

Example Code

Kitex Client

client.Option

client.WithPayloadCodec

client.WithTransportProtocol

Example Code

Use Frugal Directly

Golang Struct

Use the kitex Command Line tool

Use thriftgo (>= v0.3.0)

Manually (Not Recommended)

Encoding

Decoding

Example Code

Precautions

Slim Template

Payload Size for Decoding: Framed recommended

ARM support: not implemented yet

Performance Test Result

FAQ

How to avoid generating frugal tags?

Kitex Client Reporting “encode failed: codec msg type not match with thriftCodec”

Kitex Server Reporting “decode failed, codec msg type not match with thriftCodec”

frugal: type mismatch: 11 expected, got 10

Random stuff in the strings after decoding

Error reported compiling Kitex projects: “undefined: thrift.FrugalRead”

Optional fields are not filled with defaults with Slim template

frugal EncodeObject Reporting “unexpected EOF: 38 bytes short”

frugal EncodeObject panic

Feedback

Payload Size for Decoding: `Framed` recommended