Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat 0103 #137

Merged
merged 54 commits into from
Jan 12, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
100d2ea
Merge pull request #113 from indunet/develop
Deng-Ran Oct 8, 2022
dc5c05f
feat: add compiler
Deng-Ran Oct 10, 2022
a02db28
test: add lambda test
Deng-Ran Oct 12, 2022
15a0724
feat: lambda compiler and its tests
Deng-Ran Oct 13, 2022
a9610ef
feat: update lambda
Deng-Ran Oct 18, 2022
69412d7
doc: update readme
Deng-Ran Oct 18, 2022
a6c8ec3
fix: remove val in FormulaBuilder
Deng-Ran Oct 18, 2022
0510d53
fix: remove val in FormulaBuilder
Deng-Ran Oct 18, 2022
64a3493
build: add maven scala plugin
Deng-Ran Oct 18, 2022
cf8dd82
Merge pull request #115 from indunet/lambda
Deng-Ran Oct 18, 2022
47e1b3d
doc: update readme
Deng-Ran Oct 18, 2022
da3b000
Merge pull request #116 from indunet/lambda
Deng-Ran Oct 18, 2022
d5f255a
test: refactor fastproto benchmark
Deng-Ran Oct 20, 2022
4f34a52
test: move benchmark
Deng-Ran Oct 20, 2022
ad3dbf0
Merge pull request #117 from indunet/benchmark
Deng-Ran Oct 20, 2022
b550257
doc: add readme to benchmark
Deng-Ran Oct 20, 2022
07844cf
Merge pull request #118 from indunet/benchmark
Deng-Ran Oct 20, 2022
fddcf21
feat: add auto type
Deng-Ran Oct 21, 2022
4e85bdd
feat: update ProtocolType
Deng-Ran Oct 23, 2022
63c27a6
feat: add mapper.
Deng-Ran Oct 24, 2022
825b879
doc: add doc of auto type
Deng-Ran Oct 24, 2022
90f8451
Merge pull request #119 from indunet/autotype
Deng-Ran Oct 25, 2022
f62aa16
test: update unit test of auto type
Deng-Ran Oct 27, 2022
54e1484
feat: update api
Deng-Ran Oct 27, 2022
c727d82
Merge pull request #120 from indunet/codec
Deng-Ran Oct 27, 2022
643f56a
fix: mapper of auto type
Deng-Ran Oct 27, 2022
1cf632e
Merge pull request #121 from indunet/fix-autotype
Deng-Ran Oct 27, 2022
49c58d8
doc: add ignore chapter
Deng-Ran Nov 1, 2022
99d2831
Merge pull request #122 from indunet/doc
Deng-Ran Nov 1, 2022
fd39acc
feat: add EndianPolicy to array type
Deng-Ran Nov 1, 2022
09318bf
Merge pull request #123 from indunet/array-endian
Deng-Ran Nov 1, 2022
8425d04
doc: update readme
Deng-Ran Nov 2, 2022
2d06ade
Merge pull request #124 from indunet/doc
Deng-Ran Nov 2, 2022
e3c7ab6
doc: update manual
Deng-Ran Dec 20, 2022
bcb652a
Merge pull request #125 from indunet/doc
Deng-Ran Dec 20, 2022
3ee73b1
feat: working without annotations
Deng-Ran Dec 21, 2022
91cd369
Merge pull request #126 from indunet/feat-1221
Deng-Ran Dec 21, 2022
9ac7299
feat: add bool array type
Deng-Ran Dec 22, 2022
cbcc7df
Merge pull request #131 from indunet/feat-1222
Deng-Ran Dec 22, 2022
07b8f15
feat: update Decoder api
Deng-Ran Dec 23, 2022
d82e177
Merge pull request #132 from indunet/feat-1223
Deng-Ran Dec 23, 2022
c43e8aa
feat: update Decoder api
Deng-Ran Dec 23, 2022
0a24242
Merge pull request #133 from indunet/feat-1223
Deng-Ran Dec 23, 2022
f80210c
feat: add dynamic byte buffer
Deng-Ran Dec 25, 2022
5efcdbf
Merge pull request #134 from indunet/feat-1225
Deng-Ran Dec 25, 2022
4cd2373
feat: add AsciiType
Deng-Ran Dec 25, 2022
582168d
Merge pull request #135 from indunet/feat-1225
Deng-Ran Dec 25, 2022
ff95328
feat: add char array type
Deng-Ran Dec 29, 2022
169c786
Merge pull request #136 from indunet/feat-1229
Deng-Ran Jan 1, 2023
e5d6144
feat: add char array type
Deng-Ran Jan 3, 2023
eda01e4
doc: add annotation doc
Deng-Ran Jan 3, 2023
731fb75
feat: update dynamic byte array
Deng-Ran Jan 3, 2023
9e176a3
feat: update api
Deng-Ran Jan 4, 2023
bfa9bdf
feat: add formula to Decoder
Deng-Ran Jan 5, 2023
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc: update readme
1. update readme;
2. update unit test of lambda
  • Loading branch information
Deng-Ran committed Oct 18, 2022
commit 69412d7a232328bbb6e55f034bcbf5890bd7f540
122 changes: 78 additions & 44 deletions README-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,10 @@ FastProto是一款能够通过注解自定义协议的二进制序列化 & 反
* 支持基本数据类型、无符号类型、字符串类型、时间类型、数组类型和集合类型等
* 支持反向寻址,适用于非固定长度二进制数据
* 自定义开端字节顺序
* 自定义编码公式 & 解码公式
* 自定义编码公式 & 解码公式,支持Lambda表达式

## *Under Developing*

* 动态编译lambda形式的编码 & 解码公式
* 代码结构 & 性能优化

## *与ProtoBuf相比较*
Expand All @@ -47,7 +46,7 @@ FastProto是一款能够通过注解自定义协议的二进制序列化 & 反
<dependency>
<groupId>org.indunet</groupId>
<artifactId>fastproto</artifactId>
<version>3.6.2</version>
<version>3.7.0</version>
</dependency>
```

Expand Down Expand Up @@ -126,46 +125,18 @@ byte[] datagram = FastProto.toBytes(weather, 20);
2. **编码公式 & 解码公式**

也许你已经注意到压力信号对应一个换算公式,通常需要用户自行将序列化后的结果乘以0.1,这是物联网数据交换时极其常见的操作。
为了帮助用户减少中间步骤,FastProto引入的编码公式和解码公式。

自定义解码公式需要实现`java.lang.function.Function`接口,然后通过注解`@DecodingFormula`指定解码公式。

```java
public class PressureDecodeFormula implements Function<Long, Double> {
@Override
public Double apply(Long value) {
return value * 0.1;
}
}
```
为了帮助用户减少中间步骤,FastProto引入了编码公式注解`@EncodingFormula`和解码公式注解`@DecodingFormula`,上述简单的公式变换可以通过Lambda表达式实现。

```java
public class Weather {
...

@UInt32Type(offset = 14, decodingFormula = DecodeSpeedFormula.class)
double pressure;
}
```

同理,编码公式也需要实现`java.lang.function.Function`接口,然后注解`@EncodingFormula`指定编码公式。

```java
public class PressureEncodeFormula implements Function<Double, Long> {
@Override
public Long apply(Double value) {
return (long) (value * 10);
}
}
```
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

```java
public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(PressureDecodeFormula.class)
@EncodingFormula(PressureEncodeFormula.class)
@DecodingFormula(lambda = "x -> x * 0.1")
@EncodingFormula(lambda = "x -> (long) (x * 10)")
double pressure;
}
```
Expand Down Expand Up @@ -214,14 +185,16 @@ FastProto支持Java基础数据类型、时间类型、字符串类型、枚举
### 其它注解
FastProto还提供了一些辅助注解,帮助用户进一步自定义二进制格式、解码和编码流程。

| 注解 | 作用域 | 描述 |
|:-----------------:|:---------:|:----------:|
| @DefaultEndian | Class | 数据开端,默认小开端 |
| @DecodingIgnore | Field | 反序列化时忽略该字段 |
| @EncodingIgnore | Field | 序列化时忽略该字段 |
| @FixedLength | Class | 启动固定报文长度 |
| 注解 | 作用域 | 描述 |
|:----------------:|:---------:|:----------:|
| @DefaultEndian | Class | 数据开端,默认小开端 |
| @DecodingIgnore | Field | 反序列化时忽略该字段 |
| @EncodingIgnore | Field | 序列化时忽略该字段 |
| @FixedLength | Class | 启动固定报文长度 |
| @DecodingFormula | Field | 解码公式 |
| @EncodingFormula | Field | 编码公式 |

## 大小开端
## *大小开端*
FastProto默认使用小开端,可以通过`@DefaultEndian`注解修改全局开端类型,也可以通过endian属性修改特定字段开端,后者优先级更高。

```java
Expand All @@ -238,7 +211,68 @@ public class Weather {
}
```

## Scala
## *解码 & 编码公式*

用户可以通过两种方式自定义公式,形式较为简单的公式建议使用Lambda表达式,形式较为复杂的公式建议自定义公式类并实现`java.lang.function.Function`接口。

1. Lambda表达式

```java
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(lambda -> "x -> x * 0.1")
@EncodingFormula(lambda -> "x -> (long) (x * 10)")
double pressure;
}

```

2. 自定义公式类

```java
import java.util.function.Function;

public class PressureDecodeFormula implements Function<Long, Double> {
@Override
public Double apply(Long value) {
return value * 0.1;
}
}
```

```java
import java.util.function.Function;

public class PressureEncodeFormula implements Function<Double, Long> {
@Override
public Long apply(Double value) {
return (long) (value * 10);
}
}
```

```java
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(PressureDecodeFormula.class)
@EncodingFormula(PressureEncodeFormula.class)
double pressure;
}
```

用户可以根据需要仅指定编码公式,或者仅指定解码公式,如果同时指定Lambda表达式和自定义公式类,后者有更高的优先级。

## *Scala*
FastProto支持case class,但是Scala并不完全兼容Java注解,所以请使用如下方式引用FastProto。

```scala
Expand Down
123 changes: 79 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,10 @@ solve the problem of cross-language and cross-platform data exchange, which is e
* Support primitive type, unsigned type, string type, time type, array type and collection type
* Support reverse addressing, suitable for non-fixed length binary data
* Customize endianness (byte order)
* Support decoding formula & encoding formula
* Support decoding formula & encoding formula including lambda expression

## *Under Developing*

* Dynamically compile encoding & decoding formulas in lambda
* Code structure & performance optimization

## *Compared with ProtoBuf*
Expand All @@ -49,7 +48,7 @@ FastProto is more recommended for the following scenarios:
<dependency>
<groupId>org.indunet</groupId>
<artifactId>fastproto</artifactId>
<version>3.6.2</version>
<version>3.7.0</version>
</dependency>
```

Expand Down Expand Up @@ -133,48 +132,19 @@ byte[] datagram = FastProto.toBytes(weather, 20);

Perhaps you have noticed that the pressure signal corresponds to a conversion formula, usually requiring the user to multiply
the serialized result by 0.1, which is an extremely common operation in IoT data exchange.
To help users reduce intermediate steps, FastProto introduces encoding formulas and decoding formulas.

The custom decoding formula needs to implement the `java.lang.function.Function` interface, and then specify the decoding
formula through annotation `@DecodingFormula`
To help users reduce intermediate steps, FastProto introduces decoding formula annotation `@DecodingFormula` and encoding formula annotation `@EncodingFormula`,
the above simple formula transformation can be implemented by Lambda expression.

```java
public class PressureDecodeFormula implements Function<Long, Double> {
@Override
public Double apply(Long value) {
return value * 0.1;
}
}
```
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

```java
public class Weather {
...

@UInt32Type(offset = 14, decodingFormula = DecodeSpeedFormula.class)
double pressure;
}
```

Similarly, In the same way, the encoding formula also needs to implement the `java.lang.function.Function` interface, and
then specify the encoding formula through annotation `@EncodingFormula`.

```java
public class PressureEncodeFormula implements Function<Double, Long> {
@Override
public Long apply(Double value) {
return (long) (value * 10);
}
}
```

```java
public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(PressureDecodeFormula.class)
@EncodingFormula(PressureEncodeFormula.class)
@DecodingFormula(lambda = "x -> x * 0.1")
@EncodingFormula(lambda = "x -> (long) (x * 10)")
double pressure;
}
```
Expand Down Expand Up @@ -222,12 +192,14 @@ cross-language and cross-platform data exchange, FastProto also introduces unsig
### *Other Annotations*
FastProto also provides some auxiliary annotations to help users further customize the binary format, decoding and encoding process.

| Annotation | Scope | Description |
|:---------------:|:---------:|:-------------------------------------------------:|
| @DefaultEndian | Class | Endianness, default as little endian. |
| @DecodingIgnore | Field | Ignore the field when decoding. |
| @EncodingIgnore | Field | Ignore the field when encoding. |
| @FixedLength | Class | Enable fixed length of datagram. |
| Annotation | Scope | Description |
|:----------------:|:-----:|:-------------------------------------:|
| @DefaultEndian | Class | Endianness, default as little endian. |
| @DecodingIgnore | Field | Ignore the field when decoding. |
| @EncodingIgnore | Field | Ignore the field when encoding. |
| @FixedLength | Class | Enable fixed length of datagram. |
| @DecodingFormula | Field | Decoding formula. |
| @EncodingFormula | Field | Encoding formula. |


## Endianness
Expand All @@ -248,6 +220,69 @@ public class Weather {
}
```

## *解码 & 编码公式*

Users can customize formula in two ways. For simple formulas, it is recommended to use Lambda expression, while for more
complex formula, it is recommended to customize formula classes by implementing the `java.lang.function.Function` interface.

1. Lambda Expression

```java
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(lambda -> "x -> x * 0.1")
@EncodingFormula(lambda -> "x -> (long) (x * 10)")
double pressure;
}

```

2. Custom Formula Class

```java
import java.util.function.Function;

public class PressureDecodeFormula implements Function<Long, Double> {
@Override
public Double apply(Long value) {
return value * 0.1;
}
}
```

```java
import java.util.function.Function;

public class PressureEncodeFormula implements Function<Double, Long> {
@Override
public Long apply(Double value) {
return (long) (value * 10);
}
}
```

```java
import org.indunet.fastproto.annotation.DecodingFormula;
import org.indunet.fastproto.annotation.EncodingFormula;

public class Weather {
...

@UInt32Type(offset = 14)
@DecodingFormula(PressureDecodeFormula.class)
@EncodingFormula(PressureEncodeFormula.class)
double pressure;
}
```

Users can specify only the encoding formula or only the decoding formula as needed. If both lambda expression and custom
formula class are specified, the latter has a higher priority.

## Scala
FastProto supports case class,but Scala is not fully compatible with Java annotations, so please refer to FastProto as follows.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,5 @@

int byteOffset();

int bitOffset() default 0;
int bitOffset();
}
26 changes: 10 additions & 16 deletions src/main/java/org/indunet/fastproto/graph/resolve/CodecFlow.java
Original file line number Diff line number Diff line change
Expand Up @@ -44,33 +44,27 @@ public void process(Reference reference) {
.defaultEndianPolicy(reference.getEndianPolicy())
.build();

Function decoder = null;
if (reference.getDecodingFormulaClass() != null) {
Function decoder = CodecMapper.getDecoder(context, reference.getDecodingFormulaClass());

if (reference.getDecodingLambda() != null) {
decoder = CodecMapper.getDefaultDecoder(context, reference.getProtocolType().defaultJavaType());
} else {
decoder = CodecMapper.getDecoder(context, reference.getDecodingFormulaClass());
}

if (reference.getDecodingFormula() != null) {
val func = reference.getDecodingFormula();

reference.setDecoder(decoder.andThen(func));
reference.setDecoder(decoder);
} else if (reference.getDecodingLambda() != null) {
val decoder = CodecMapper.getDefaultDecoder(context, reference.getProtocolType().defaultJavaType());
val func = reference.getDecodingLambda();

reference.setDecoder(decoder.andThen(func));
} else {
Function decoder = CodecMapper.getDecoder(context, null);

reference.setDecoder(decoder);
}

if (reference.getEncodingFormula() != null) {
val encoder = CodecMapper.getDefaultEncoder(context, reference.getProtocolType().defaultJavaType());
val func = reference.getEncodingFormula();
if (reference.getEncodingFormulaClass() != null) {
val encoder = CodecMapper.getEncoder(context, reference.getEncodingFormulaClass());

reference.setEncoder((byte[] bytes, Object value) -> encoder.accept(bytes, func.apply(value)));
reference.setEncoder(encoder);
} else if (reference.getEncodingLambda() != null) {
val encoder = CodecMapper.getEncoder(context, reference.getEncodingFormulaClass());
val encoder = CodecMapper.getDefaultEncoder(context, reference.getProtocolType().defaultJavaType());
val func = reference.getEncodingLambda();

reference.setEncoder((byte[] bytes, Object value) -> encoder.accept(bytes, func.apply(value)));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ public void process(Reference reference) {
throw new FormulaException(String.format("fail initializing formula %s", clazz.getSimpleName()), e);
}
} else if (!formula.lambda().isEmpty()) {
val inputType = reference.getProtocolType().defaultJavaType();
val inputType = reference.getField().getType();
val builder = FormulaBuilder.create(inputType, formula.lambda());

reference.setEncodingLambda(builder.build());
Expand Down