如何将 Keras 合并层用于具有两个输出的自动编码器

Question

假设我有两个输入：XandY我想设计和联合自动编码器来重建X'and Y'。如图所示，X是音频输入，Y是视频输入。这种深层架构很酷，因为它有两个输入和两个输出。此外，它们共享中间的某个层。我的问题是如何使用Keras来编写这个自动编码器。假设除了中间的共享层之外，每一层都是全连接的。这是我的代码如下： from keras.layers import Input, Dense from keras.models import Model import numpy as np X = np.random.random((1000, 100)) y = np.random.random((1000, 300))  # x and y can be different size # the X autoencoder layer  Xinput = Input(shape=(100,)) encoded = Dense(50, activation='relu')(Xinput) encoded = Dense(20, activation='relu')(encoded) encoded = Dense(15, activation='relu')(encoded) decoded = Dense(20, activation='relu')(encoded) decoded = Dense(50, activation='relu')(decoded) decoded = Dense(100, activation='relu')(decoded) # the Y autoencoder layer  Yinput = Input(shape=(300,)) encoded = Dense(120, activation='relu')(Yinput) encoded = Dense(50, activation='relu')(encoded) encoded = Dense(15, activation='relu')(encoded) decoded = Dense(50, activation='relu')(encoded) decoded = Dense(120, activation='relu')(decoded) decoded = Dense(300, activation='relu')(decoded)我只是中间有和 的15节点。我的问题是如何用损失函数训练这个联合自动编码器？XY\|X-X'\|^2 + \|Y-Y'\|^2

Answer 1

您的代码方式是您有两个单独的模型。虽然您可以简单地将共享表示层的输出用于以下两个子网，但您必须合并这两个子网作为输入：

Xinput = Input(shape=(100,))

Yinput = Input(shape=(300,))

Xencoded = Dense(50, activation='relu')(Xinput)

Xencoded = Dense(20, activation='relu')(Xencoded)

Yencoded = Dense(120, activation='relu')(Yinput)

Yencoded = Dense(50, activation='relu')(Yencoded)

shared_input = Concatenate()([Xencoded, Yencoded])

shared_output = Dense(15, activation='relu')(shared_input)

Xdecoded = Dense(20, activation='relu')(shared_output)

Xdecoded = Dense(50, activation='relu')(Xdecoded)

Xdecoded = Dense(100, activation='relu')(Xdecoded)

Ydecoded = Dense(50, activation='relu')(shared_output)

Ydecoded = Dense(120, activation='relu')(Ydecoded)

Ydecoded = Dense(300, activation='relu')(Ydecoded)

现在你有两个独立的输出。所以你需要两个单独的损失函数，无论如何都会添加它们来编译模型：

model = Model([Xinput, Yinput], [Xdecoded, Ydecoded])

model.compile(optimizer='adam', loss=['mse', 'mse'], loss_weights=[1., 1.])

然后，您可以通过以下方式简单地训练模型：

model.fit([X_input, Y_input], [X_label, Y_label])