flink model serving 增加支持pytorch功能

Deep Java Library (DJL) is an open-source, high-level, engine-agnostic Java framework for deep learning. DJL is designed to be easy to get started with and simple to use for Java developers. DJL provides a native Java development experience and functions like any other regular Java library.


You don't have to be machine learning/deep learning expert to get started. You can use your existing Java expertise as an on-ramp to learn and use machine learning and deep learning. You can use your favorite IDE to build, train, and deploy your models. DJL makes it easy to integrate these models with your Java applications.


Because DJL is deep learning engine agnostic, you don't have to make a choice between engines when creating your projects. You can switch engines at any point. To ensure the best performance, DJL also provides automatic CPU/GPU choice based on hardware configuration.


DJL's ergonomic API interface is designed to guide you with best practices to accomplish deep learning tasks. The following pseudocode demonstrates running inference:


    // Assume user uses a pre-trained model from model zoo, they just need to load it
    Criteria<Image, Classifications> criteria =
            Criteria.builder()
                    .optApplication(Application.CV.OBJECT_DETECTION) // find object dection model
                    .setTypes(Image.class, Classifications.class) // define input and output
                    .optFilter("backbone", "resnet50") // choose network architecture
                    .build();


    try (ZooModel<Image, Classifications> model = ModelZoo.loadModel(criteria)) {
        try (Predictor<Image, Classifications> predictor = model.newPredictor()) {
            Image img = ImageFactory.getInstance().fromUrl("http://..."); // read image
            Classifications result = predictor.predict(img);


            // get the classification and probability
            ...
        }
    }
The following pseudocode demonstrates running training:


    // Construct your neural network with built-in blocks
    Block block = new Mlp(28, 28);


    try (Model model = Model.newInstance("mlp")) { // Create an empty model
        model.setBlock(block); // set neural network to model


        // Get training and validation dataset (MNIST dataset)
        Dataset trainingSet = new Mnist.Builder().setUsage(Usage.TRAIN) ... .build();
        Dataset validateSet = new Mnist.Builder().setUsage(Usage.TEST) ... .build();


        // Setup training configurations, such as Initializer, Optimizer, Loss ...
        TrainingConfig config = setupTrainingConfig();
        try (Trainer trainer = model.newTrainer(config)) {
            /*
             * Configure input shape based on dataset to initialize the trainer.
             * 1st axis is batch axis, we can use 1 for initialization.
             * MNIST is 28x28 grayscale image and pre processed into 28 * 28 NDArray.
             */
            Shape inputShape = new Shape(1, 28 * 28);
            trainer.initialize(new Shape[] {inputShape});


            EasyTrain.fit(trainer, epoch, trainingSet, validateSet);
        }


        // Save the model
        model.save(modelDir, "mlp");
    }

Open Neural Network Exchange (ONNX) is an open ecosystem that empowers AI developers
 to choose the right tools as their project evolves. ONNX provides an open source format
  for AI models, both deep learning and traditional ML. It defines an extensible computation 
  graph model, as well as definitions of built-in operators and standard data types. Currently
   we focus on the capabilities needed for inferencing (scoring).


ONNX is widely supported and can be found in many frameworks, 
tools, and hardware. Enabling interoperability between different
 frameworks and streamlining the path from research to production 
 helps increase the speed of innovation in the AI community. 
 We invite the community to join us and further evolve ONNX.

model_inits = {
    'rf_lw50_voc'   : rf_lw50, 
}
n_classes=2
models = dict()
for key,fun in six.iteritems(model_inits):
    # 这里调用的其实是def rf_lw50(num_classes, imagenet=False, pretrained=True, **kwargs)
    net = fun(n_classes, pretrained=True).eval()
    if has_cuda:
        net = net.cuda()
    models[key] = net
    model=net
#给定一个确定的输入 实际上可以随机初始化一个和需要的尺寸一致的numpy
img_path="/home/ahhh/PycharmProjects/light-weight-refinenet/examples/imgs/blind/1110a.png"
img = np.array(Image.open(img_path))
input =  (torch.Tensor(prepare_img(img).transpose(2, 0, 1)[None])).float()
if has_cuda:
    input = input.cuda()
# onnx模型文件输出
torch_out = torch.onnx._export(model, input, "lw50.onnx",export_params=True)

https://github.com/microsoft/onnxruntime-openenclave/blob/openenclave-public/java/src/test/java/ai/onnxruntime/InferenceTest.java#L66